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1.  INTRODUCTION 


SRI  International's  Decision  Analysis  Group  is  conducting  research  on 
decision  structuring  for  the  Operational  Decision  Aids  (ODA)  Program  of  the 
Office  of  Naval  Research.  The  goal  of  SRI's  research  effort  is  to  develop  a 
computer-assisted  decision  structuring  process  for  use  by  a naval  task  force 
commander  (TFC)  and  his  staff  and  to  implement  the  associated  computer  aids 
on  the  ODA  test  bed.  Research  to  date  lias  resulted  in  the  design  and 
partial  computer  implementation  of  a decision  structuring  procedure.  The 
structuring  procedure  and  recent  progress  toward  completing  its 
implementation  are  described  in  this  report. 

1 . 1 ODA  Program  Background 

The  ODA  Program  was  initiated  in  1974  as  an  interdisciplinary  effort 
designed  to  apply  advances  in  four  professional  areas  to  the  development  of 
navy  command  and  control  systems.  The  four  areas  contributing  to  the  effort 
have  been  computer  science,  decision  analysis,  systems  analysis,  and 
organizational  psychology.  The  effort  has  focused  on  developing  decision 
aids  for  application  to  decisions  at  the  level  of  the  TFC,  and  the  approach 
has  been  to  contract  for  the  development  of  specific  products  from  a variety 
of  contractors. 
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In  addition  to  the  SRI  decision  structuring  aid,  decision  aids  already 
developed  or  currently  being  developed  within  the  Program  include  aids  that 
address  information  processing  [1,2,3],  man-machine  communication  [4,5], 
military  outcome  estimation  [6,7,8],  and  nomography  and  uncertainty  analysis 
[9].  As  the  various  aids  are  developed,  they  are  being  programmed  and 
installed  on  a test  bed  in  the  Department  of  Decision  Sciences  of  the 
Wharton  School,  University  of  Pennsylvania.  The  purpose  of  the  test  bed  has 
been  to  provide  a central  experimental  site  for  aid  evaluation.  An 
underlying  assumption  of  the  research  program  has  been  that  all  products 
should  be  subjected  to  objective  experimental  testing  to  evaluate  their 
usefulness  as  operational  decision  aids  for  naval  decision  making. 

1 . 2 Need  for  Structuring  Aids 

Decision  structuring  is  the  process  of  identifying  and  organizing  the 
factors  of  a decision  in  such  a way  that  logic  may  be  applied  to  identify  a 
preferred  decision  strategy.  Decision  structuring  is  usually  easy  for  the 
TFC.  Most  of  his  decisions  are  familiar  and  readily  dealt  with  by  existing 
personal  and  organizational  problem-solving  mechanisms.  Occasionally, 
however,  the  TFC  will  encounter  an  important  decision  that  is  difficult 
because  of  its  complexity,  its  unfamiliarity,  or  both.  When  little  time  is 
available  for  deciding,  this  can  create  stress  that  significantly  degrades 
decision-making  ability.  For  these  situations,  aids  are  needed  to  help  the 
TFC  structure  relevant  information  into  a formal  model  whose  analysis  will 
help  identify  the  most  desirable  course  of  action. 
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1.2.1  Prestructured  Models 


One  approach  for  reducing  the  problems  associated  with  unstructured 
decisions  Is  to  provide  the  TFC  with  prestructured  models.  In  a 
prestructured  nodel,  the  relevant  factors  and  relationships  for  a specific 
class  of  situations  that  may  be  encountered  are  identified  and  modeled  in 
advance.  These  models  are  then  programmed  on  a computer  to  provide  the  TFC 
with  a convenient  means  for  simulating  the  outcomes  of  alternative  courses 
of  action.  In  some  implementations,  the  models  can  be  used  to  optimize  an 
action  according  to  some  criterion  function. 

Many  of  the  decision  aids  currently  under  development  within  the  ODA 
Program  can  be  regarded  as  prestructured  models.  These  aids  attempt  to 
capture  the  Issues  surrounding  one  particular  class  of  TFC  decision 
problem — for  example,  airstrike  timing  emission  control  planning,  transit 
route  planning,  and  so  forth. 


Prestructured  models 
establishing  an  organized 
situations,  they  decrease 
situation  for  which  he  is 


can  be  valuable  decision  aids  for  the  TFC.  By 
structure  for  analyzing  certain  decision 
the  likelihood  that  the  commander  will  encounter 
totally  unprepared. 


a 


There  are,  however,  certain  risks  associated  with  providing  the  TFC 
with  prestructured  models  only.  Because  the  models  are  applicable  only  to 
the  specific  situations  for  which  they  were  designed,  any  decisions  that  are 
not  covered  by  the  models  must  necessarily  be  made  without  the  benefit  of 
decision  aids.  Furthermore,  using  prestructured  models  as  decision  aids  can 
present  difficulties  even  when  the  models  are  applied  to  the  situations  for 
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which  they  wore  designed.  The  commander  may  lack  confidence  that  the 
prestructured  model  accurately  reflects  all  of  the  unique  considerations  of 
his  current  situation.  The  commander's  problem,  then,  is  how  to  account  for 
those  aspects  of  a situation  that  may  not  be  accurately  reflected  by  the 
prestructured  model.  Fortunately  structuring  aids  can  reduce  the  risks  and 
Increase  the  usefulness  of  prestructured  models. 

1.2.2  Structuring  Aids  Complement  Prestructured  Models 

Decision  structuring  aids  complement  prestructured  models.  They  can  be 
applied  when  prestructured  models  cannot,  and  they  can  be  used  in 
conjunction  with  prestructured  models  to  produce  integrated  decision  aids. 

When  the  commander  is  confronted  with  an  unstructured  decision  problem 
that  is  not  easily  dealt  with  using  prestructured  models  or  other  standard 
methods  of  analysis,  lack  of  time  and  resources  usually  prevents  the 
development  of  anything  more  sophisticated  than  a qualitative  or  extremely 
simple  quantitative  analysis.  The  value  of  a decision  structuring  aid  is 
that  it  expands  the  commander's  capability  to  construct  rapidly  a formal 
quantitative  model  that,  like  a properly  designed  prestructured  model, 
accurately  accounts  for  all  of  the  relevant  considerations  of  his  problem. 
Unlike  prestrur tured  models,  however,  a structuring  aid  enables  the  decision 
maker  to  construct  a decision  model  in  real  time,  after  the  characteristics 
of  the  situation  have  been  identified. 

The  ability  to  structure  a decision  model  in  real  time  provides  several 
important  advantages.  First,  if  the  decision  maker  recognizes  factors  in 
the  current  situation  that  are  not  accounted  for  by  the  prestructured 
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models,  those  factors  can  be  incorporated  into  the  decision  process  by 
structuring  a decision  model  that  includes  both  the  relevant  prestructured 
models  and  those  factors  that  are  not  included  in  the  models.  Second,  the 
structuring  process  can  be  used  to  develop  decision  models  for  unanticipated 
situations  for  which  no  prestructured  models  have  been  supplied.  Third,  a 
decision  structuring  process  allows  the  decision  maker  to  participate 
directly  in  the  construction  of  the  decision  model,  thereby  providing  him 
with  a better  understanding  of  the  model's  asumptions  and  limitations. 
Because  of  this,  the  decision  maker  is  likely  to  have  confidence  in  the 
model  and  is  more  likely  to  use  it  effectively. 

1 . 3 Decision  Tree  Structuring  Format 

The  SRI  decision  structuring  process  is  a computer-aided  procedure  for 
rapidly  constructing  and  analyzing  a decision  model.  The  form  of  the 
decision  model  produced  is  a decision  tree.  The  decision  tree  format  was 
selected  because  it  is  a very  general  model  form  whose  characteristics  make 
it  well  suited  for  representing  complex,  sequential  decisions  that  deal  with 
uncertainty . 

Figure  1 illustrates  a decision  tree  model  for  a TFC  planning  decision 
taken  from  the  ONRODA  Scenario.*  According  to  the  scenario,  a Blue  TFC  has 
been  assigned  the  mission  of  neutralizing  an  enemy  attack  anticipated  to  be 
launched  the  next  morning  from  ONRODA  airfield  against  a Blue  ally.  The 


*"The  ONRODA  Warfare  Scenario  [10]"  describes  the  recent  political  and  mili- 
tary history  leading  to  a potential  military  conflict  involving  the  hypothet- 
ical countries  of  Blue,  Red,  Grey,  and  Orange  and  the  Island  of  ONRODA.  This 
scenario  has  been  designed  to  be  used  by  contractors  to  test  and  to  illus- 
trate decision  aids  proposed  within  the  ODA  program. 
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Figure  1 DECISION  TREF.  REPRESENTING  DECISION  TAKEN 
FROM  THE  ONRODA  SCENARIO 
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decision  Is  whether  the  Blue  TFC  should  plan  to  neutralize  the  ONRODA 
airfield  with  an  alrstrlke  or  through  a blockade.  Important  considerations 
for  this  decision,  represented  In  the  tree.  Include  whether  Red  has  civilian 
aircraft  (such  as  hospital  planes)  on  the  airfield  when  the  commander 
launches  the  alrstrlke,  and  whether  Red  will  attack  the  task  force  in 
retaliation  for  the  commander's  actions. 

In  the  decision  tree  format,  decisions  are  represented  by  small 
squares,  called  decision  nodes,  with  the  various  alternatives  shown  as  lines 
emanating  from  each  square.  Uncertainties,  such  as  enemy  actions,  are 
represented  in  the  diagram  by  small  circles,  called  chance  nodes.  Tin* 
branches  emanating  from  chance  nodes  represent  the  possible  outcomes  to  each 
uncertain  event. 

The  decision  and  chance  nodes  in  a decision  tree  are  arranged  according 
to  the  sequence  In  which  decisions  must  he  made  and  in  which  outcomes  of 
uncertain  events  will  he  revealed  to  the  decision  maker.  Thus,  each  path 
leading  from  left  to  right  through  the  tree  represents  a different  possible 
sequence  of  decisions  and  events  that  might  occur.  As  an  example,  suppose 
the  commander  chooses  to  plan  an  alrstrlke.  The  decision  tree  shows  that 
reconnaissance  will  indicate  whether  Red  has  hospital  planes  on  the  airfield 
before  the  attack.  If  hospital  planes  are  sighted,  the  commander  can  either 
launch  his  alrstrlke  or  revert  to  a blockade.  Following  whatever  action  Is 
chosen.  Red  may  or  may  not  attack  the  task  force. 

A decision  tree  can  be  solved  to  obtain  an  optimal  decision  strategy. 
The  first  step  is  to  provide  the  necessary  numerical  inputs — probability 
assessments  for  the  likelihood  of  the  various  outcomes  to  uncertain  events, 
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and  value  assessments  representing  the  relative  desirability  of  each  path 
through  the  tree.*  Figure  1 shows  example  probabilities  under  the  branches 
from  chance  nodes  and  sample  values  at  the  terminal  points  of  the  paths 
through  the  tree. 

The  next  step  is  to  compute  expected  values  for  each  node  in  the  tree. 
Expected  values  can  be  obtained  using  a roll-back  procdure.  The  expected 
value  at  a chance  node  may  be  obtained  by  multiplying  each  terminal  value  by 
the  probability  along  the  branch  leading  to  that  value,  and  adding. 

Similarly  if  the  rule  is  used  that  the  decision  maker  wili  prefer  the 
decision  alternative  with  the  highest  expected  value,  the  expected  values 
for  decision  nodes  can  be  determined. 


Rolling  expected  values  back  through  the  decision  tree  in  this  manner 
enables  a preferred  decision  strategy  to  be  determined.  The  preferred 
decision  strategy  is  the  path  through  the  tree  that  leads  through 
alternatives  with  the  highest  (or  least  negative)  expected  values.  For  the 
example,  the  computed  expected  values  indicate  that  the  optimal  initial 
decision  is  to  plan  an  airstrike.  The  optimal  decision  strategy  is  to 
conduct  an  airstrike  if  no  Red  hospital  planes  are  sighted  and  to  revert  to 
a blockade  if  Red  hospital  planes  are  sighted. 


There  are  a number  of  advantages  to  selecting  decision  trees  as  the 
model  for  decision  structuring.  Most  importantly,  as  mentioned  earlier,  the 


*The  values  assigned  to  terminal  nodes  represent  the  desirability  of  the  out- 
comes corresponding  to  each  path  through  the  tree.  If  a decision  maker's 
risk  aversion  is  also  assessed,  these  values  can  be  converted  to  utilities 
that  account  for  attitude  toward  risk  taking.  Although  we  have  chosen  not  to 
consider  risk  attitude  in  this  report,  all  results  can  be  easily  extended  to 
reflect  risk  aversion. 
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decision  tree  is  a general  model  form  that  can  be  used  to  represent  a wide 
class  of  decision  situations.  Another  advantage  is  that  a decision 
structuring  process  based  on  decision  trees  is  compatible  with  the  use  of 
prestructured  models.  Prestructured  models  can  be  used  for  simplifying  the 
generation  of  numerical  inputs.  Specifically,  models  can  be  developed  to 
help  estimate  the  probabilities  associated  with  chance  nodes,  the  outcomes 
associated  with  each  path  through  a decision  tree,  and  the  values  to  be 
assigned  to  the  terminal  points  of  each  path  through  the  tree.  For  example 
if  an  airstrike  outcome  calculator  were  available  for  estimating  the 
military  outcomes  under  various  strike  patterns,  it  could  be  used  in 
conjunction  with  the  decision  tree  in  Figure  1 to  produce  an  integrated 
dec  is  ion  model . 

Another  major  advantage  to  structuring  a decision  as  a decision  tree  is 
that  the  methods  for  analyzing  and  solving  decision  trees  are  easily 
programmed.  Thus,  the  same  computer  aid  that  helps  construct  the  decision 
tree  can  be  used  to  solve  it.  More  importantly  for  a decision  structuring 
process,  a computer  can  be  used  to  analyze  a decision  tree  structure  to 
identify  how  that  structure  should  be  expanded  so  as  to  improve  its 
rel lability . 

1 . 4 Research  Strategy  and  Current  Status 

Tills  report  is  the  third  in  a series  of  reports  documenting  SRI's 
research  progress  toward  developing  a computer-aided  decision  structuring 
process.  Since  the  beginning  of  the  research  effort,  an  adaptive  research 
strategy  has  been  applied.  As  a result,  the  current  direction  and  goals  of 
the  effort  have  been  strongly  influenced  by  earlier  results  and  conclusions. 
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In  recognition  of  the  usefulness  of  decision  trees  for  representing 
decision  structure,  initial  research  focused  on  formalizing  the  process  by 
which  experienced  analysts  construct  decision  tree  models.  Several 
experimental  structuring  sessions  were  recorded,  transcribed,  and  then 
analyzed  with  the  objective  of  Identifying  effective  procedures  for 
eliciting  from  a decision  maker  a decision  tree  representation  of  his 
situation.  The  major  result  was  specification  of  the  elemental  steps 
performed  as  a decision  analyst  structures  a decision  model.  The  research, 
which  is  summarized  in  Reference  [11].  showed  that  the  process  of  building  a 
decision  tree  model  could  be  represented  as  a sequential  application  of 
these  elemental  steps. 

The  next  phase  of  the  research  was  an  evaluation  of  the  degree  to  which 
a protocol  for  ordering  the  elemental  structuring  steps  would  aid  the 
development  of  a decision  model.  The  evaluation,  which  included  several 
applications  to  current  and  past  decision  problems,  indicated  that  the 
structuring  protocol  provided  useful  guidance  for  translating  known  decision 
factors  into  a mathematical  model.  However,  the  major  limitation  of  the 
structuring  protocol  was  that  it  failed  to  provide  guidance  for  identifying 
the  factors  that  should  be  included  in  the  analysis.  Two  other  important 
limitations  of  the  standard  method  of  analysis  were  also  apparent. 
Application  of  decision  analysis  procedures  frequently  results  in 
considerable  effort  being  spent  to  model  aspects  of  the  decision  problem 
that  have  little  or  no  impact  on  the  decision.  Furthermore,  the  standard 
method  of  analysis  provides  little  indication  of  which  alternative  being 
analyzed  is  best  until  the  final  stages  of  the  analysis.  This  can  be  a 
serious  drawback  if,  as  occasionally  happens,  the  user  must  terminate 
analyses  prematurely. 
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As  a result  of  these  conclusions,  research  was  directed  toward 


developing  a structuring  process  that  both  guides  the  modeling  effort  by 

I 

helping  the  user  to  Identify  factors  likely  to  impact  the  decision  and 
supports  a method  of  analysis  that  quickly  produces  a tentative  solution, 
which  may  be  improved  through  additional  modeling.  The  resulting  process  Is 
called  the  decision  tree  expansion  algorithm  because  it  is  based  on  the 
concept  of  progressively  expanding  a decision  tree — an  initial  decision  tree 
model  is  analyzed  to  identify  those  areas  of  the  model  where  adding  more 
detail  is  most  likely  to  indicate  an  improved  decision.  The  decision  tree 
expansion  algorithm  and  a preliminary  computer  Implementation  are  described 
in  Reference  [12]. 

Experimental  applications  of  the  decision  tree  expansion  algorithm 
showed  that  two  major  Improvements  would  be  needed  to  increase  its 
usefulness  as  a decision  aid.  First,  the  computer  implementation  would  have 
to  be  made  more  convenient  and  flexible  by  both  streamlining  the  algorithm 
and  providing  graphics  for  output  display.  Second,  procedures  were  needed 
to  help  the  user  develop  the  preliminary  decision  tree  used  to  initiate  the 
tree  expansion  algorithm.  The  most  recent  stage  of  the  research  has  been 
directed  at  these  two  areas.  Consequently,  although  tills  report  summarizes 
the  current  status  of  the  SRI  decision  structuring  process,  the  major 
emphasis  is  on  refinements  in  the  design  of  the  expansion  algorithm  that 
improve  its  convenience  and  flexibility,  improved  graphics  design  for  the 
user  interface,  and  the  preliminary  structuring  process  that  has  been 
designed  to  help  the  user  develop  the  initial  decision  tree  required  to 
enter  the  expansion  algorithm. 


1 1 


The  research  strategy  for  completing  the  development  of  the  computer- 
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OVERVIEW  OF  THE  DECISION  STRUCTURING  AID 


This  section  presents  an  overview  of  the  structuring  process  and  Its 
associated  computer  aids.  A more  detailed  description  of  the  process  Is 
presented  In  Sections  3 and  4. 

2. 1 Major  Components  and  Important  Characteristics 

Figure  2 Illustrates  the  major  components  of  the  SRI  decision 
structuring  process.  The  process  Is  carried  out  in  two  distinct  phases. 

The  first  phase,  called  the  preliminary  structuring  phase,  is  designed  to 
develop  a simple  decision  tree  that  represents  only  those  factors  and 
relationships  immediately  apparent  to  the  decision  maker.  This  initial 
decision  tree  is  referred  to  as  the  preliminary  decision  model.  The  primary 
aid  for  the  preliminary  structuring  phase  is  an  interactive  computer  program 
for  systematic  inquiry  and  decision  tree  generation. 

Because  the  first  simple  model  that  is  constructed  often  overlooks 
factors,  the  preliminary  structuring  phase  is  followed  by  a second  phase, 
called  the  expansion  phase.  The  purpose  of  the  expansion  phase  is  to 
identify  and  bring  into  the  preliminary  decision  model  additional  important 
factors.  The  primary  aid  for  the  expansion  phase  is  an  interactive  computer 
program  for  decision  tree  expansion  and  analysis. 
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Figure  2 MAJOR  COMPONENTS  OF  THE  STRUCTURING  PROCESS 


As  will  become  clear  in  Section  4,  application  of  the  structuring 
process  requires  the  user  to  supply  quantitative  estimates  for  the  outcomes 
to  each  decision  alternative  under  various  assumptions.  Providing  the  large 
number  of  outcome  estimates  can  become  burdensome  if  calculated  by  hand;  so, 
the  structuring  process  is  designed  to  be  used  with  prestructured  models  for 
estimating  military  outcomes.  In  the  preliminary  structuring  phase,  a 
simulation  map  is  used  as  an  auxiliary  aid  for  estimating  decision  outcomes. 
In  the  expansion  phase,  an  analogous  but  more  sophisticated  version  of  the 
same  aid  is  an  outcome  calculator.  The  structuring  process  may  be  used 
without  prestructured  models  for  estimating  outcomes;  however,  its  use  in 
this  mode  will  be  more  time-consuming. 

The  ultimate  output  of  the  structuring  process  is  a fully  structured 
decision  model  along  with  the  insights  and  solutions  produced  through  an 
analysis  of  that  model.  Interim  outputs  include  a series  of  preliminary 
models  and  the  solutions  to  these  models.  Since  the  decision  model  is 
improved  as  it  is  expanded  to  include  more  and  more  of  the  relevant  decision 
factors,  the  preliminary  solutions  are  improved  as  the  process  is  carried 
out.  Having  a tentative  solution  always  available  is  valuable,  because  the 
structuring  process  may  be  profitably  terminated  at  any  point.  The  most 
recent  model  and  its  solution  provide  an  approximation  to  the  optimal 
decision  strategy  that  would  be  produced  if  the  structuring  process  were 
carried  out  to  completion.  In  practice,  the  decision  maker  will  terminate 
the  structuring  process  whenever  he  perceives  that  the  improvements  gained 
from  further  structuring  and  analysis  do  not  warrant  additional  Investments 
of  his  time. 
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Additional  features  that  enhance  the  value  of  the  structuring  process 
as  a decision  aid  Include  the  following: 

• The  applicability  of  the  aid  is  exceedingly  general.  It  may  bo  used 
to  structure  virtually  any  dynamic,  uncertain,  decision  problem  with 
single  or  multiple  objectives.  There  is  no  limit  to  the  number  of 
alternatives  or  uncertain  decision  factors  that  can  be  included  in 
the  decision  structure. 

• The  aid  does  not  require  common  over-simpl i f icat ions  concerning  the 
probabilistic  independence  of  uncertainties  or  the  shapes  of  their 
probability  distributions. 

• The  aid  tests  new  factors  introduced  during  the  expansion  phase  for 
relevance  to  the  decision  at  hand,  thereby  avoiding  time-consuming 
modeling  effort  when  it  lias  no  impact  on  the  decision. 

• The  algorithm  provides  a method  for  identifying  events  that  are  of 
critical  concern  to  the  TFC.  Such  events  could  be  used  as  triggers 
(for  example,  setting  alerts  in  the  DAISY  System). 

• Application  of  the  aid  produces  graphical  output  that  summarizes 
decision  structure  and  documents  the  decision-making  process. 

2.2  T es_t  Facility  Conf lgurat ion  and  Graphical  Capabilities 

The  Interactive  computer  programs  for  applying  the  decision  structuring 
process  have  been  designed  and  are  partially  implemented  on  the  ODA  test-bed 
facility.  Figure  3 shows  the  configuration.  Three  video  screens  may  be  used 
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Figure  3 DEVICES  USED  IN  TEST-BED  IMPLEMENTATION 


simultaneously  for  output:  a color  monitor  tied  to  a Grlnnel  display  system 
for  representing  decision  trees  and  Influence  diagrams,  a CONAPL  graphics 
terminal  for  alphanumeric  input  and  output,  and  a Tektronix  4013  for 
plotting  probability  distributions  and  sensitivity  plots.  Input  devices 
consist  of  a keyboard  and  track  ball. 

Figures  4 through  6 give  examples  of  the  display  formats  designed  for 
the  various  outputs  produced  during  application  of  the  structuring  process. 
Figure  4 shows  the  use  of  the  CONAPL  graphics  as  the  primary  input/output 
interface.  Here,  the  computer  is  requesting  outcome  estimates  required  for 
the  expansion  phase.  All  computer  generated  prompts  and  questions,  as  well 
as  user  alphanumeric  input,  are  displayed  on  the  CONAPL.  Figure  5 is  a plot 
of  a probability  distribution  that  the  computer  has  fitted  to  the  user's 
high,  best,  and  low  estimates  for  aircraft  losses.  Probability 
distributions  and  sensitivity  plots  are  displayed  on  the  Tektronix.  Figure  6 
illustrates  the  manner  in  which  a decision  tree  is  displayed  on  the  Grinnel 
monitor.  Although  not  visible  in  the  black  and  white  illustration,  decision 
trees  and  other  model  representations  are  displayed  using  color.  For 
example,  when  a decision  tree  is  displayed,  the  decision  strategy  with  the 
highest  expected  value  is  denoted  with  a blue  arrow  and  the  path  through  the 
tree  recommended  for  additional  structuring  is  highlighted  in  red.  Most  of 
the  alphanumeric  and  graphical  displays  provided  by  the  aid  are  intended  for 
use  by  a staff  analyst  and  other  staff  officers  who  provide  inputs  to  the 
system.  Some  displays,  however,  are  designed  to  be  viewed  by  the  commander 
as  well.  These  include  displays  showing  sensitivities,  preference 
relationships,  and  recommended  decision  strategies. 
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Figure  4 TYPICAL  ALPHANUMERIC  DISPLAY  USED 
IN  STRUCTURING  PROCESS 
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2 . 3 Implementation  as  a Man-Machine  System 


Figure  7 illustrates  how  the  decision  structuring  process  is  designed 
to  be  implemented  as  a man-machine  system.  The  commander  works  with  a 
member  of  his  staff,  henceforth  referred  to  as  the  staff  analyst,  to 
identify  and  organize  the  relevant  factors  of  his  decision  into  a decision 
model.  The  computer  assists  the  staff  analyst  by  acting  as  a prompting 
system,  as  a medium  for  constructing  the  decision  model,  and  as  a tool  for 
assisting  the  evaluation  of  alternatives. 

From  the  commander's  point  of  view,  the  structuring  activity  consists 
of  a well-directed  dialogue  with  his  staff  analyst.  The  staff  analyst  asks 
specific  questions  regarding  the  commander's  assessment  of  the  situation  and 
suggests  issues  for  detailed  discussion.  The  commander,  in  turn,  is 
responsible  for  providing  (or  verifying)  the  judgmental  inputs  required  for 
the  analysis.  The  computer  prompts  the  staff  analyst  with  an  orderly 
sequence  of  questions  and  data  requests  for  building  a decision  tree  model. 
The  staff  analyst  enters  the  required  data,  translating  or  interpreting  . 
information  provided  by  the  commander,  if  necessary. 

As  the  decision  model  is  constructed,  the  computer  analyzes  the  model 
to  identify  sensitive  model  areas  that  should  be  refined  or  expanded. 
Normally,  the  commander  will  not  operate  the  computer  directly.  However, 
depending  on  the  commander's  wishes,  the  staff  analyst  may  supply  the 
commander  witti  a summary  or  detailed  "running  report"  of  analysis  results. 

Elements  of  the  decision  model  that  are  not  specific  to  the  unique 
characteristics  of  the  decision  at  hand  are  provided  as  inputs  to  the 
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computer.  One  major  input  required  is  a value  model  that  describes  the 
relative  importance  or  value  of  the  attributes  of  military  outcomes — for 
example,  the  willingness  to  trade  off  own  force  losses  of  various  types  for 
accomplishment  of  mission  objectives.  Other  major  inputs  are  prestructured 
models  for  estimating  military  outcomes.  As  mentioned  above,  use  of 
prestructured  outcome  calculators  allows  the  structuring  process  to  be 
carried  out  in  the  most  rapid  and  efficient  manner  possible. 


3-  THE  PRELIMINARY  STRUCTURING  PHASE 


As  discussed  briefly  in  Section  2,  the  SRI  decision  structuring  process 
consists  of  a preliminary  structuring  phase  and  an  expansion  phase.  The 
preliminary  structuring  phase  is  designed  to  produce  a simple  decision  tree 
model  representing  only  the  most  important  factors  of  the  decision  as 
initially  perceived  by  the  decision  maker.  Although  the  preliminary 
structuring  phase  precedes  the  expansion  phase  in  a typical  application,  the 
historical  development  of  the  two  phases  proceeded  in  the  opposite  order. 

The  decision  tree  expansion  algorithm  underlying  the  current  expansion 
phase  was  originally  viewed  as  the  basis  for  the  entire  structuring  process. 

The  original  concept  of  operation  was  that  a user  would  define  a "zero' th 
order  tree"  consisting  only  of  a single  decision  node,  with  branches 
eminating  from  that  node  representing  each  of  the  recognized  alternatives. 

The  expansion  algorithm  would  then  be  used  to  expand  the  tree,  adding  one 
factor  at  a time,  until  a decision  tree  was  obtained  that  contained  all  of 
the  major  uncertainties  and  down-stream  decisions  required  to  represent 
accurately  the  decision  situation. 

• j 

■ 

Early,  informal  applications  of  the  expansion  algorithm  pointed  out 
that,  although  this  approach  worked,  it  was  an  inefficient  way  to  structure 
those  decision  factors  already  known  to  the  decision  maker.  A procedure  was 
needed  for  quickly  generating  an  initial  decision  tree  that  contained  those 
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decision  factors  and  relationships  that  are  Immediately  apparent. 
Consequently,  research  was  Initiated  to  develop  a preliminary  structuring 
process . 

3. 1 Concept  of  Operation 

The  preliminary  structuring  process  Is  an  elicitation  procedure 
designed  to  clarify  the  commander's  understanding  of  his  decision  problem 
and  to  assess  the  Information  necessary  to  construct  a decision  tree 
representing  the  most  important  of  the  readily  apparent  decision  factors. 

The  elicitation  is  conducted  through  a series  of  questions  grouped  according 
to  the  specific  function  is  being  carried  out  in  the  structuring  process. 
Each  function  accomplishes  a necessary  step  in  the  construction  of  a 
decision  tree.  For  example,  one  structuring  function  is  designed  to  elicit 
the  set  of  feasible  alternative  courses  of  action  available  to  the 
commander.  Another  function  is  designed  to  specify  a set  of  uncertainties 
that  must  be  accounted  for  in  the  selection  of  a decision  strategy. 

The  number  and  conteiU  of  the  specific  questions  composing  each 
function  are  adapt tve  and  depend  on  the  answers  given  to  previous  questions. 
In  addition,  the  order  in  which  the  questions  are  posed  and,  more  generally, 
the  overall  approach  are  designed  to  achieve  four  important  objectives: 

1.  Structuring  activity  Initially  focuses  on  improving  the 

understanding  of  decision  objectives  and  then  on  refining  an 
initially  preferred  alternative.  The  intent  is  to  begin  producing 
results  directly  applicable  to  the  formation  of  a decision  strategy 
as  quickly  as  possible. 


2.  Input  requirements  are  kept  to  a minimum.  The  process  Is  designed 
primarily  to  organize  and  simplify  the  application  of  human 
problem-solving  skills  rather  than  to  replace  those  skills  with 
machine  processing.  This  contributes  to  relatively  speedy 
generation  of  insights. 

3.  The  steps  and  specific  questions  asked  are  designed  to  encourage  a 
more  objective  understanding  of  the  situation  by  countering  several 
recognized  decision-making  errors  and  biases. 

4.  The  output  of  the  process  is  a decision  tree  in  a format  consistent 
with  the  requirements  of  the  expansion  phase. 

Figure  8 is  a flowchart  showing  the  specific  preliminary  structuring 
functions  and  the  order  in  which  they  are  executed.  The  purpose  of  each  of 
the  structuring  functions  as  well  as  motivation  for  the  design  are 
summarized  in  the  paragraphs  below-  The  detailed  steps  and  queries  for 
carrying  out  each  function  are  described  in  Subsection  3.3. 

3.1.1  Objective  and  Outcome  Variable  Identification 

The  first  function  to  be  executed  in  the  preliminary  structuring 
process  is  objective  and  outcome  variable  identification.  The  principal 
outputs  are  a list  of  decision  objectives  and  a list  of  associated  outcome 
variables. 

The  initial  steps  in  this  function  elicit  a complete  set  of  objectives 
for  the  decision.  The  elicitation  is  designed  to  identify  not  only  physical 
but  any  political,  economic,  or  other  military  decision  objectives  as  well. 
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For  example,  one  objective  for  a naval  mission  will  often  be  to  achieve  some 
physical  objective,  such  as  gaining  control  of  a geographical  area.  Another 
objective  will  be  to  minimize  own  force  losses.  Still  another  might  be  to 
avoid  any  action  that  might  provoke  another  nation  to  enter  the  conflict  as 
an  ally  of  the  enemy. 

Once  a complete  list  of  objectives  is  identified,  the  next  step  is  to 
create  a list  of  outcome  variables.  Outcome  variables  are  measures  for 
assessing  the  degree  to  which  the  decision  outcome  achieves  each  objective. 
The  outcome  variables  should  be  quantifiable — quantities  whose  values  may 
be  expressed  by  numbers.  For  example,  outcome  variables  for  the  objective 
"minimize  own  force  loses"  might  be  the  number  and  type  of  own  force  units 
that  are  destroyed  or  sustain  significant  damage  as  a result  of  the  decision 
outcome • 

Research  suggests  that  a structuring  process  designed  to  focus 
Initially  on  objectives  rather  than  alternatives  has  some  advantages.  A 
common  decision-malting  error  is  to  define  t lie  decision  problem  too  narrowly 
by  confusing  a specific  alternative  with  an  objective.  For  example,  suppose 
a TFC  finds  himself  in  a situation  where  he  must  protect  a group  of 
civilians  on  a small  island  who  are  threatened  by  an  anticipated  invasion. 
The  commander  may  perceive  his  decision  problem  as  a choice  between 
strategies  for  preventing  the  invasion.  By  stepping  back  from  his  problem 
and  recognizing  that  his  objective  Is  to  protect  civilians,  he  is  less 
likely  to  overlook  other  viable  alternatives,  such  as  evacuating  the 
civilians  from  the  island.  Focusing  first  on  the  objectives  of  the  decision 
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tends  to  clarify  the  decision  maker's  understanding  of  the  situation  and 
promotes  openmindedness  in  the  generation  of  alternatives. 

3.1.2  Alternative  Identification 

The  second  function  in  the  preliminary  structuring  process  is 
elicitation  of  decision  alternatives.  This  is  obviously  a crucial  step 
because  no  amount  of  analysis  can  lead  to  the  selection  of  a best 
alternative  if  that  alternative  has  not  been  included  in  the  decision 
st  rue  ture . 

Unfortunately  little  is  known  about  the  creative  process  of  defining 
new  alternatives.  Limited  research  suggests  that  the  creation  of  new 
alternatives  can  be  encouraged  by  involving  several  individuals  with 
different  points  of  view  in  the  decision-making  process,  by  applying 
brainstorming  techniques,  and  by  using  morphological  methods.  Few  of  these 
techniques,  however,  have  been  developed  to  the  point  that  they  can  be 
Incorporated  into  an  automated  decision  aid. 

The  alternative  Identification  function  in  the  preliminary  structuring 
process  makes  use  of  two  techniques  for  encouraging  the  generation  of  a 
complete  list  of  alternatives.  The  first  is  the  formalization  of  a "devil's 
advocate"  role  for  challenging  each  decision  alternative.  This  is  discussed 
below  in  the  description  of  the  scenario  generation  function.  The  second 
technique  is  to  supply  the  commander  with  a list  of  generic  alternatives. 

The  list  of  generic  alternatives  used  in  this  function  is  derived  from  a 
study  of  historic  decision  situations  conducted  by  CAC1,  Inc.  (13].  By 
considering  each  generic  decision  type  in  the  list,  the  commander  can 
decrease  the  likelihood  that  an  effective  alternative  has  been  overlooked. 
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The  output  of  the  alternative  generation  function  is  a list  of  feasible 
courses  of  action. 

3.1.3  Alternative  Evaluation 

The  alternative  evaluation  function  is  designed  to  produce  an  estimate 
of  the  likely  outcomes  and  risks  associated  with  each  alternative  course  of 
action.  Scenario  generation  is  used  as  the  basis  for  the  evaluation. 

Because  of  their  training  and  experience,  TFC's  tend  to  think  in  terms  of 
events  and  reactions  to  events.  Thus,  scenarios  provide  a familiar, 
concise,  and  consistent  framework  for  organizing  the  commander's  thoughts. 

Steps  in  the  function  are  designed  to  permit  the  TFC  and  the  staff  to 
generate  scenarios  that  describe  the  likely  outcomes  of  a given  course  of 
action  as  well  as  scenarios  that  make  the  course  of  action  look  very  bad. 
Generating  a scenario  that  makes  a course  of  action  look  very  bad 
accomplishes  two  things.  First,  it  counteracts  a general  error  or  bias  in 
decision  making  sometimes  referred  to  as  "mindguards" . Mindguards  refers  to 
the  observed  tendency  of  decision-makers  to  limit  or  distort  their  world 
views  so  as  to  produce  a less  threatening  perception  of  difficult 
situations.  Typical  examples  of  such  behavior  are  enemy  stereotyping  and 
rationalization.  The  result  of  these  biases  is  usually  a failure  to 
consider  adequately  the  risks  or  negative  aspects  of  a prefered  course  of 
action.  By  formalizing  a devil's  advocate  evaluation  of  each  alternative,  a 
deliberate  attempt  is  made  to  counteract  the  tendency  to  create  mindguards. 

A second  value  to  generating  a scenario  that  makes  each  alternative  look 
very  bad  is  that  consideration  of  possible  risks  challenges  the  commander 
and  his  staff  to  come  up  with  modifications  to  the  course  of  action  or  new 
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alternatives  that  achieve  the  same  benefits  without  the  risks.  An  earlier 


phase  of  the  SRI  research  effort  indicated  that  challenging  a decision  maker 
by  pointing  out  specific  disadvantages  or  risks  associated  with  a prefered 
action  is  an  effective  way  to  encourage  the  creation  of  new  alternatives. 

For  this  reason,  as  shown  in  Figure  8.  the  output  of  the  alternative 
evaluation  function  can  lead  back  into  the  alternative  generation  function. 

3.1.4  Critical  Uncertainty  Identification 

The  critical  uncertainty  identification  function  is  designed  to  produce 
a list  of  crucial  state  variables  representing  those  uncertainties  whose 
probabilities  must  be  explicitly  accounted  for  in  the  decision  model.  The 
approach  to  generating  the  list  is  to  compare  the  most  likely  and  worst 
scenarios  for  each  retained  alternative.  The  difference  in  the  scenarios 
represents  uncertain  events  that  influence  the  relative  preference  for  the 
al ternatives . 

Once  the  uncertainties  that  influence  the  decision  are  identified,  each 
uncertainty  is  described  by  a state  variable  defined  so  that  the  possible 
values  for  the  variable  correspond  to  the  possible  outcomes  of  the  uncertain 
event.  For  example,  if  the  difference  between  a likely  and  worst  possible 
scenario  for  a given  alternative  is  the  prevailing  weather  condition 
assumed,  weather  would  be  listed  as  a critical  uncertainty.  A corresponding 
state  variable  could  be  defined  by  specifying  a variable  called  weather  that 
could  take  one  of  two  values:  good  or  bad.  For  simplicity,  all  uncertain 
variables  defined  in  this  structuring  function  are  assumed  to  be  discrete. 


■* 
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3.1.5  Sequence  and  Cause/Effect  Relationship  Specification 


The  sequence  and  cause/effect  relationship  specification  function  is 
designed  to  produce  a summary  of  the  relationships  among  decision  variables, 
uncertain  state  variables,  and  outcome  variables.  This  is  accomplished  by 
using  influence  diagrams.  Influence  diagrams  are  diagrams  whose  structure 
summarizes  the  probabilistic  dependencies  among  uncertainties  and  specifies 
the  sequence  in  which  decisions  must  be  made  and  information  becomes 
available.  The  specific  steps  of  this  function  that  permit  the  construction 
of  an  influence  diagram  are  illustrated  in  Section  3.3. 

3.1.6  Preliminary  Decision  Tree  Construction 

The  final  function  in  the  preliminary  structuring  phase  is  to  construct 
and  evaluate  a preliminary  decision  tree.  Steps  in  this  function  convert 
che  influence  diagram  created  in  the  previous  function  into  a decision  tree 
structure.  The  tree  structure  is  the  major  output  of  the  preliminary 
structuring  phase  and  the  major  input  to  the  expansion  phase.  The  required 
numerical  inputs  for  the  tree — value  assignments  and  probabilities — are 
elicited  and  the  tree  is  solved  in  the  expansion  phase. 

3.2  Aids  for  Preliminary  Structuring 

The  primary  computer  aid  for  the  preliminary  structuring  phase  is  an 
interactive  program  for  systematic  inquiry  and  decision  tree  construction. 

An  important  associate  aid  used  to  support  outcome  estimation  and  detailed 
planning  is  a simulation  map. 


3 . 2 . i Interactive  Program  for  Systematic  Inquiry  and  Decision  Tree 
Construction 

The  program  for  inquiry  and  decision  tree  construction  consists  of  two 
main  elements:  an  automated  questionnaire  that  guides  the  user  through  the 
six  preliminary  structuring  functions  and  an  influence  diagram  manipulator 
used  to  simplify  the  construction  of  a decision  tree.  Both  elements  have 
been  designed,  but  neither  has  been  implemented  on  the  ODA  Program  test  bed 
The  Influence  diagram  manipulator,  however,  has  been  partially  implemented 
on  computers  at  SRI  International. 

Figure  9 illustrates  the  use  of  the  graphics  terminal  as  an  automated 
questionnaire.  The  first  few  questions  that  step  the  user  through  the 
objective  identification  function  are  shown  together  with  sample  user 
responses.  The  complete  set  of  questions  for  the  automated  questionnaire  1 
Illustrated  In  Subsection  3-3. 

The  primary  motivation  for  automating  the  questionnaire  is  to  permit 
automatic  adjustment  of  "prompting  detail  and  branching.  Three  levels  of 
prompting  detail  are  available,  ranging  from  a detailed  request  with 
explanation  for  each  input  required  to  abbreviated  prompting.  Branching 
provides  a way  to  tailor  the  questionnaire  to  the  user's  needs  In  a 
particular  decision  situation.  As  an  example,  proper  execution  of  the 
objective  and  outcome  variable  ident i f ica t ion  function  should  generate  a 
small  list  of  separable  outcome  measures.  If  too  many  outcome  measures  are 
defined,  later  input  requirements  are  increased.  Furthermore,  some  of  the 
measures  are  likely  to  be  redundant  or  of. only  minor  importance.  The 
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Figure  9 AUTOMATED  QUESTIONNAIRE 


computer,  therefore,  counts  the  number  of  outcome  measures  that  have  been 
identified.  If  there  are  less  than  five,  no  additional  action  is  taken;  if 
there  are  more  than  five,  a series  of  questions  is  presented  to  identify  and 
eliminate  less  significant  measures  and  to  combine  outcome  measures  that  are 
redundant.  , 

The  influence  diagram  manipulator  allows  the  user  to  construct,  modify, 
and  analyze  influence  diagrams.  Influence  diagrams  are  graphical  devices 
for  describing  the  relationships  among  the  variables  in  a decision.  They 
are  a more  general  representation  than  decision  trees  and,  yet,  are  easily 
interpreted  by  individuals  at  all  levels  of  technical  proficiency.  Methods 
for  converting  influence  diagrams  to  decision  trees  can  be  automated. 

Hence,  influence  diagrams  provide  a useful  device  for  simplifying  the 
construction  of  a decision  tree.  A detailed  description  of  the  theory  and 
use  of  Influence  diagrams  is  contained  in  Reference  [14]. 

The  influence  diagram  manipulator  permits  the  user  to  constuct  and 
display  an  influence  diagram  with  the  support  of  the  computer.  It  is  used 
along  with  the  automated  questionnaire  to  simplify  execution  of  the  sequence 
and  cause/effect  specif ication  function  and  the  decision  tree  construction 
function.  Figure  10  illustrates  the  use  of  the  influence  diagram 
manipulator  in  conjunction  with  a question  from  the  sequence  and 
cause/effect  specification  function.  Boxes  with  right-angled  corners  in  the 
influence  diagram  represent  decisions.  Boxes  with  rounded  corners  are 
chance  nodes  and  represent  critical  uncertainties.  An  arrow  from  a chance 
node  or  a decision  node  to  another  chance  node  indicates  that  the 
probabilities  associated  with  the  second  uncertainty  depend  on  the  outcome 
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SEQUENCE  AND  CAUSE/EFFECT  RELATIONSHIPS 


3 DECISIONS  AND  CRITICAL  UNCERTAIN  EVENTS  ARE  REPRESENTED 
BELOW.  DRAW  ARROWS  BETWEEN  THE  BOXES  SO  AS  TO  DESCRIBE  THE 
DIRECTION  OF  INFLUENCES  (CONSTRUCT  THE  INFLUENCE  DIAGRAM) 


of  the  first  uncertainty  or  the  decision  taken.  An  arrow  from  a chance  node 
or  a decision  node  to  another  decision  node  indicates  that  the  second 
decision  Is  made  with  knowledge  of  the  first  uncertain  outcome  or  decision. 

3.2.2  Simulation  Map 

The  simulation  map  Is  a manual  war  game  for  simulating  the  force 
movements  and  engagement  outcomes  resulting  from  a given  course  of  action. 
The  game  concept  is  similar  to  a simulation  game  recently  developed  at  the 
Naval  War  College  [15]  and  other  commercially  available  war  games  (see,  for 
example.  Reference  [16]). 

Figure  11  shows  the  playing  surface  for  the  simulation  map  used  in 
sample  applications  to  decisions  from  the  ONRODA  Scenario.  The  surface  is 
roughly  three  by  five  feet  with  geographic  boundaries  drawn  to  scale  and 
covered  by  a grid.  Playing  pieces  are  provided  to  represent  each  force  unit 
that  may  potentially  become  involved  in  the  conflict.  The  game  rules 
specify  constraints  on  the  movement  of  the  game  pieces  and  provide 
guidelines  for  determining  the  likely  positions  of  force  units  and  the 
losses  resulting  from  a given  own  course  of  action  and  assumed  enemy 
response . 

Figure  12  shows  several  of  the  playing  pieces  used  in  sample 
applications.  Each  playing  piece  is  a small  cardboard  square  coded  to 
represent  the  movement  capabilities  and  offensive  and  defensive  strengths  of 
a particular  force  unit.  Uncertainty  in  detection  and  kill  is  explicitly 
represented  in  estimated  outcomes.  The  various  force  strength  numbers  on 
each  playing  piece  are  selected  to  reflect  appropriate  exchange  ratios. 
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SIMULATION  MAP 


Base  Location 


Anti-Air  Strength 
Anti-Surface  Strength 
Anti-Sub  Strength 
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Figure  12  TYPICAL  SIMULATION  PLAYING  PIECES 


(Matching  the  offensive  and  defensive  strengths  of  opposing  units  produces 
engagement  outcome  probabilities  that  are  consistent  with  published  exehang 
ratios.)  A method  of  table  look  up  is  used  to  obtain  probabilities  of 
possible  outcomes  for  each  engagement. 

The  simulation  map  is  used  in  the  alternative  evaluation  function  to 
help  estimate  decision  outcomes.  The  user  moves  the  game  pieces 
corresponding  -to  his  forces  to  reflect  a given  course  of  action.  He  then 
moves  enemy  pieces  according  to  likely  enemy  responses  and  observes  the 
resulting  engagement  outcomes.  In  practice,  the  user  will  make  small 
adjustments  to  each  proposed  strategy  so  that,  when  he  plays  it  out  'on  the 
simulation  map,  he  minimizes  Ills  likely  losses  for  a given  level  of 
e f f ec  t iveness  . 

The  simulation  map  provides  sufficient  detail  to  model  realistically 
the  complex  interactions  of  an  engagement  between  opposing  naval  forces: 
yet,  it  is  flexible  enough  to  accommodate  several  different  scenarios 
without  extensive  redesign.  Although  the  simulation  map  is  currently 
Implemented  as  a manual  system,  it  could  obviously  be  fully  automated. 

3. 3 Sample  Application  of  the  Preliminary  Structuring  Aid 

The  detailed  questions  and  method  of  application  for  the  preliminary 
structuring  functions  are  best  illustrated  through  the  results  of  a sample 
application.  Exhibit  1 Illustrates  the  preliminary  structuring  questions 
generated  and  user  responses  obtained  during  an  application  to  the  planning 
decision  in  the  ONRODA  Scenario.  For  clarification,  the  dialogue  has  been 
outlined  and  labeled  to  show  at  each  point  which  preliminary  structuring 
function  is  being  executed.  Machine-generated  questions  and  prompts  are 
shown  in  bold-face  type.  User  responses  are  shown  in  italics. 
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EXHIBIT  1: 


SAMPLE  APPLICATION— PRELIMINARY  STRUCTURING  PHASE 


A.  Objectives  and  Outcome  Identification 


1.  What  is  the  incentive  or  reason  for  the  assigned  task? 

Ally  Grey  htif?  been  attacked  by  enemy  Orange.  Survival  of  Grey 
governs  tent  threatened  by  further  Or  an  ye  aggression. 

2.  What  is  your  superior's  objective? 

Support  Grey  in  its  efforts  to  defeat  the  Greyhauks  and  to 
resist  an  invasion  by  Orange. 

3.  What  is  the  objective  of  the  task  force?  if  the  mission 
has  multiple  objecti  ves , what  are  the  priori  ties? 

m order  of  importance:  1.  Neutralise  the  Orange  air  forces 

on  ONRODA  Island. 

Avoid  direct  involvement  in  con- 
flict by  Red. 

S.  Deliver  military  supplies  to  Grey. 

4.  What  specific  measures,  could  be  used  to  indicate  the  degree 
of  mission  success? 

Number  of  sorties  enemy  can  launch  aaainst  Greu  followina 
Blue  action. 


Probability  of  Blue /Red  war. 

[If  the  user  indicates  he  is  having  trouble  with  this  question 
(e.g.,  if  lie  types  :iri  : )y  the  program  moves  immediately  to  item 
A3.  After  a response  is  given,  question  A4  reappears  in  the 
modified  form: 


4'.  Are  there  any  additional  outcome  measures  that  are  specific 
to  this  mission?  If  so,  please  list  them.) 

3.  The  following  are  outcome  measures  that  may  be  important  for 
judging  the  success  ofnaval  task  force  mission’s . Check  any 
additional  measures  that,  if  not  explicitly  considered,  mijjht 
resul  t in  life  choice- of  I poor  al  ternaITve . 


time  requ i red  to,  complete  mission 

■tu-.r  own  equipment  damaged  or  destroyed 

.eQuipnent  .damaged  or  destroyed 

position  or  capabilities  of  forces  at  conclusion  or 

engagement 

political  conseguences 


*Three  levels  of  detail  are  available  in  the  elicitation  format. 
Level  1 (most  detail)  is  shown.  Level  2 consists  of  only  underlined 
questions.  Level  3 consists  of  only  double  underlined  questions. 
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6.  Your  answers  indicate  the  following  preliminary  hierarchy 
of  objectives: 

Superior's  objectives: 

Support  Grey  in  its  efforts  to  defeat  the  Grey- 

hawks  and  to  resist  an  invasion  by  Orange. 

Task  force  objectives: 

A.  Neutralize  the  Orange  air  forces  on  ONRODA 
Island 

B.  Avoid  direct  involvement  in  conflict  by  Red 

C.  Deliver  military  supplies  to  Grey 

Outcome  measures: 

1.  Number  of  sorties  enemy  can  launch  against 
Grey  following  Blue  action 

2.  Probability  of  Blue/Red  war 

3.  Own  equipment  damaged  or  destroyed 

Are  all  aspects  of  the  task  force  objectives  accounted  for 
by  the  outcome  measures?  If  not,  please  identify  those 
aspects  that  are  not  currently  included. 

(© 

[If  tlie  user  indicated  instead  that  something  has  been  omitted, 
the  following  version  of  question  A4  is  repeated: 

bn . (displayed  only  if  response  to  item  6 is  positive)  What 
specific  measures  could  be  used  to  indicate  the  degree  to 
which  this  objective  is  achieved? 

Any  newly  identified  outcome  measures  are  added  to  the  displayed 
list.] 

[If  more  than  a total  of  5 outcome  measures  are  listed,  there  is 
a good  chance  that  some  are  redundant  and  the  following  questions 
are  asked : 

6a’.  (displayed  only  if  the  number  of  outcome  measures  is  greater 
than  5)  Consider  each  outcome  measure  in  turn.  If  you 
ignored  this  aspect  of  the  outcome,  could  it  possibly  lead 


you 

to  an  incorrect  decision? 

Important  to 

Decision 

Outcome  Measure 

Yes 

No 

1. 

Number  of  sorties  enemy  can  launch 
against  Grey  following  Blue  action 

XXX 

2. 

Probability  of  Blue/Red  war 

XXX 

3. 

Own  equipment  damaged  or  destroyed 

XXX 

4. 

Enemy  aircraft  destroyed 

XXX 

5. 

Tons  of  military  supplies  to  Grey 

XXX 
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Table  1 

KEY  TO  SYMBOLS  USED  IN  DISPLAYS 


Blue,  Grey, 

Orange , Red 

ships : 

<e) 

<$> 

Blue,  Grey, 

Orange,  Red 

aircraft : 

(S\ 

m A 

A 

Orange , Red 

submarines : 

V/ 

\R/ 
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6b'.  (displayed  only  if  the  number  of  outcome  measures  is 

greater  than  5)  Are  any  of  the  outcome  measures  redundant? 
Two  outcome  measures  are  redundant  if  knowing  the  level  of 
one  essentially  tells  you  the  level  of  the  other. 

Yes 

Which  outcome  measures  are  redundant  (identify  by  number). 


I > 4 

These  outcome  measures  will  be  combined.  What  do  you  wish 
to  call  the  result? 

Enemy  sortie  capability \ 

Alternative  Generation 

(Alternative  generation  and  alternative  evaluation  are  accomplished 
with  the  aid  of  the  simulation  map  (described  in  Subsection  3.1.2). 
Displays  1 through  14  included  in  this  exhibit  are  rough  sketches 
showing  the  approximate  configuration  of  the  game  pieces  and  the 
movement  of  game  pieces  at  various  stages  of  preliminary  structuring 
process.  The  key  defining  display  symbols  is  given  in  Table  1.  Dis- 
play 1 shows  the  configuration  of  the  simulation  map  at  the  initiation 
of  this  function.] 

l.  Prepared  list  oftentati ve^own  courses  of  action  which,  if 
successful , wi 1 1 accompli sh  the  mission . 

— Airs  trike  of  OilRODA  followed  with  Orange /Oil RODA  blockade 

— Destroy  ONRODA  airfield  with  naval  gunfire 

— Amphibious  landing 

— Grey /ONRODA  air  blockade 

[If  the  user  indicates  that  he  is  having  difficulty  compiling  a 
list  of  alternatives  (e.g.,  if  he  types  HELP),  the  following  is 
displayed  and  item  B1  is  then  repeated: 

l'.  Examples  of  own  courses  of  action  include: 

--blockade  or  military  quarantine 

--commit  sea  forces  to  combat 

--commit  air  forces  to  combat 

--show  of  military  force 

--airlift  personnel  or  supplies 

--threaten  to,  or  actually,  withdraw  support 

--reposition  forces 

--commit  support  services 

--conduct  military  maneuvers  or  training  exercises 
--improve  force  readiness 
--provide  supplies] 

[The  simulation  board  is  now  altered  to  show  the  detailed  loca- 
tions and  types  of  all  relevant  friendly  and  enemy  forces 
(Display  2) . ] 


SPLAY  1:  SIMULATION  MAP 


LOCATION'S  OF  FORCE  UNITS 


C . Alternative  Evaluation — Airstrike 


1.  Select  the  course  of  action  which  appears  to  have  the  best 
chance  for  success.  Trace  out  planned  movement  on  the 
simulation  board. 

[Airstrike  is  tentatively  selected.  Planned  movement  is  traced 
out  by  the  commander  Display  3)  is  as  follows: 

— Air  defense  unit  (lxFL4)*  returned  to  CV1  and  other  air 
defense  unit  (lxFl4)  returned  to  CV2 

— CV2  and  support  ships  moved  to  position  250nmi  west  of  ONRODA 
— DLG  to  engage  Orange  surface  defense  (lxMB) 
on  northwest  side  of  ONRODA 
— CLC  to  engage  Orange  surface  defense  (lxMB) 
on  west  side  of  ONRODA 

— D1)G  and  CG  set  up  surface  defense  midway  between  ONRODA  and 
Industrial  City 

— CV1  sends  air  strike  (lxF14,  lxA7,  lxAb)  to  attack  ONRODA 
from  north 

— CV2  sends  air  strike  (lxF14,  2xA7)  to  attack  ONRODA  from  east 
and  intercept  any  Orange  reinforcements  arriving  from  Orange 
Port  and  Pier  City 

Logic  built  into  the  simulation  board  checks  the  intended  move- 
ment pattern  for  feasibility  in  view  of  time/distance  factors, 
transit  speeds,  movement  constraints,  and  offensive  and  defensive 
strength  capabilities.  An  automated  version  of  the  board  would 
actually  show  the  pieces  moving  to  highlight  forced  deviations 
from  the  intended  movement  pattern.  Display  4 show:’  the  locations 
achieved  by  the  intended  movement  pattern.] 

2.  Trace  out  anticipated  enemy  response  movement  on  simulation 
board . 

[I'n  it  movement  anticipated  for  Red  and  Orange  as  input  by  the 
commander  is  shown  in  Figure  5.  Basic  movement  assumptions  are: 

— Red  does  not  attack  the  task  force 

— Air  reinforcements  (2xMlC21  2xMlC19)  are  brought  to  ONRODA 
from  Pier  City  and  Orange  Port 

— Surface  defense  (4xTB,  2xMB)  arrive  and  join  existing  defense 
(2xMB) 

— Surviving  aircraft  from  attack  on  Grey  attempt  to  return  to 
ONRODA 

— Orange  launches  an  air  defense  between  ONRODA  and  CV2  (3xMIG21) 
witli  the  intent  of  intercepting  and  preventing  the  return  of 
Blue  attack  aircraft  to  CV2  . | 


*The  symbols  "F14,"  "Ab,"  etc.  denote  squadrons  of  aircraft. 
12  planes  to  a squadron.  Thus,  "2xF14,"  for  example,  denotes 
rons  totalling  24  planes. 
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There  are 
two  squad- 


friendly  force  unit  locations 


IMDOi 


DISPLAY  5:  ANTICIPATED  ENEMY  MOVEMENTS,  AIRSTRIKE 


[Given  positioning  and  intended  movement  patterns,  outcome  esti- 
mation logic  built  into  simulation  hoard  estimates  enemy  losses 
and  generates  a probability  tree  describing  possible  Blue  losses 
resulting  from  combat.  Losses  can  result  from  probability  of 
kill  factors  relating  to  the  engagement  of  opposing  units  or,  in 
the  case  of  aircraft,  due  to  insufficient  fuel  to  permit  a safe 
return  to  a carrier  or  land  airbase. 

Simulation  board  logic  first  simulates  the  attempted  return  to 
ONKODA  of  the  Orange  aircraft  sent  to  attack  Grey.  It  is  estimated 
that  the  Orange  attack  wave  sent  to  Industrial  City  incurs  heavy 
losses  against  Grey  base  defense  (1xM1G21,  lxMIG19,  lxSU7)  and 
against  Grey  air  defense  and  Blue  CAP  (lxSU7,  1xM1G21),  as  does 
tlie  attack  on  Grey  Port  (lxMIG21,  lxMIG19,  lxTU16,  lxSU7).  Remain- 
ing aircraft  (lxMIG21,  lxSU7 , 2xlL28,  lxTUlb)  successfully  dis- 
engage and  begin  return  to  ONRODA.  Display  6 shows  the  unit  loca- 
tions as  indicated  by  the  simulation  board  at  this  time. 

ONRODA  combat  is  next  simulated.  Heavy  Orange  losses  are  indicated 
(6xMIC21,  lxMIG19,  lxSU7 , 2xlL28) . Blue  losses  are  estimated  for 
best,  nominal  and  worse  case  outcomes.  Losses  occur  either  as  a 
direct  result  of  the  Blue  attack  or  due  to  Orange  counter  attacks 
as  input  by  the  user.  The  table  below  summarizes  total  Orange  and 
Blue  losses: 


ESTIMATED  ENGAGEMENT  LOSSES 
AIRSTR1KE  UNDER  ANTICIPATED  ENEMY  RESPONSE 

Orange:  9xMIG21,  3xMIC19,  4xSU7 , 2xlL28,  lxTU16 
Blue:  Aircraft  Ships 


Minimum:  0.1xF14, 

3xA7 , 

A6 

0 

Nominal:  0.9xF14, 

3xA7, 

A6 

0 

Maximum:  1.6xF14, 

3::A7 , 

A6 

0 

As  in  the  case  of  Orange  losses  incurred  in  the  attack  on  Grey, 
the  simulation  board  indicates  the  damage  incurred  during  the 
ONRODA  attack  to  specific  units.  Under  the  nominal  case,  for 
example,  the  user  observes  that  his  attack  from  the  east  suffers 
heavy  damage  (0.9xF14,  2xA7)  and  is  not  successful  in  preventing 
resupply  of  Orange  aircraft  to  ONRODA  from  Orange  Port.) 
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DISPLAY  6:  FORCE  UNIT  LOCATIONS  FOLLOWING  ANTICIPATED  ENEMY  MOVEMENTS,  AIRSTRIKE 


3 . Do  you  wish  to  revise  the  planned  movement  for  this  course 
of  action  or  the  assumed  enemy  response?  If  so,  make  the 
desired  revisions . 

[Noting  that  the  squadron  of  F14  suffers  heavy  losses  partially 
due  to  the  Orange  missile  boats  anticipated  to  the  east  of  ONRODA 
and  that  he  is  not  able  to  prevent  reinforcement  from  Orange  Port, 
the  user  elects  to  modify  his  planned  attack  to  allow  the  second 
squadron  of  F14s  to  attack  from  the  west  rather  than  the  east  of 
ONRODA.  Display  7 shows  the  revised  attack  plan,  the  intended 
movement  pattern  is  identical  to  that  sIiowti  in  Figure  3 except 
that  the  second  squadron  of  F14s  attacks  from  the  west.  No 
changes  are  made  to  the  anticipated  Orange  action. 

Simulation  of  the  attack  now  indicates  30%  damage  to  the  second 
squadron  of  F14s  in  the  nominal  case.  Other  losses  remain  the 
same.  Orange  losses  and  total  minimum,  nominal,  and  maximum  Blue 
losses  are  now  estimated  by  the  simulation  board  to  be: 


ESTIMATED  ENGAGEMENT  LOSSES— REVISED 
A1RSTRIKE  UNDER  ANTICIPATED  ENEMY  RESPONSE 


Orange:  9xMIG21,  3xMIGl9,  4xSU7 , 2xIL28,  lxTU16 


Blue:  Aircraft Ships 


Minimum:  0.1xF14,  3xA7,  A6  0 
Nominal:  0.3xF14,  3xA7,  A6  0 
Maximum:  lxF14,  3xA7,  A6  0] 


4.  Bear  in  mind  that  the  results  produced  by  the  simulation 

board  are  accurate  only  to  the  extent  that  the  board  reflects 
your  current  situation.  Taking  into  account  all  aspects  of 
your  situation,  evaluate  the  alternative  according  to  each  of 
the  outcome  measures. 

Alternative:  Airstrike  on  ONRODA  followed  with  Orange/ 

ONRODA  blockade— anticipated  enemy  response 


Outcome  Measure 

Enemy  capability  to 
launch  sorties  against 
Grey  following  Blue 
Action 

Risk  of  Blue/Red  War 

Own  equipment  damaged  or 
destroyed 


Ev alu lation 

Orange  sustains  heavy 
damage  to  their  aircraft . 
Unlikely  that  they  would 
attempt  to  launch  another 
attack  against  Grey. 

High 

Low  to  moderate 
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DISPLAY  7:  PLANNED  MOVEMENT,  REVISED  AIRSTRIKE 


5.  Taking  the  point  of  view  of  the  enemy  commander,  trace  out 
conceivable  enemy  courses  of  action  that  make  the  alternative 
look  very  bad. 

[The  commander ' s main  concern  is  his  vulnerability  to  attack. 

He  first  considers  the  possibility  of  an  Orange  attack  on  the 
task  force. 

The  Orange  movement  anticipated  by  the  commander  in  the  event 
of  an  attack  is  illustrated  in  Display  8 in  the  form  in  which  it 
would  be  input  to  the  simulation  board.  Movement  assumptions  are 
the  same  as  in  the  previous  case  except  that  Orange  launches  an 
attack  against  CV2  (6xMIG21,  1XIL28) . Positions  of  opposing 
forces  (after  accounting  for  movement  constraints  and  attrition 
of  Orange  aircraft  during  return  from  attack  on  Grey)  are  shown  in 
Display  9. 

As  in  the  anticipated  enemy  response  case,  the  outcome  estimation 
Togic  of  the  simulation  hoard  is  used  to  generate  a nominal  esti- 
mate for  Orange  losses  and  minimum,  nominal,  and  worse-case  Blue 
losses.  Estimated  losses  are: 


ESTIMATED  ENGAGEMENT  LOSSES — REVISED 
AIRSTRIKE  UNDER  WORSE-CASE  ENEMY  RESPONSE  It  1 

Orange:  llxMIC2l,  3xMIG19,  4xSU7 , 2xIL28,  lxTU16 


Aircraft Shijgs 

Minimum:  0.1xF14,  3xA7,  A6  0 
Nominal:  0.6xF14,  3xA7 , 1.3A6  0 
Maximum:  2xF14,  3xA7 , 2A6  0 


Contrary  to  the  commander's  initial  impression,  the  simulation 
board  does  not  indicate  significantly  greater  Blue  losses  in  the 
event  of  an  Orange  attack  under  the  airstrike  alternative.  Com- 
paring the  above  results  with  the  estimated  losses  under  the 
anticipated  enemy  response  indicates,  if  Orange  attacks,  that  Orange 
can  expect  to  lose  two  additional  squadrons  of  MIG21s  while  Blue 
can  expect  to  lose  an  additional  two  tenths  squadron  of  F14s . 
or  about  two  planes.  The  explanation  for  this  is  that  the  comman- 
der has  left  one  squadron  each  of  F14s  , and  A6s,  to  defend  CV2 . 
According  to  probabi lity-of-kill  ratios,  these  planes  together 
with  the  antiair  capability  of  the  ships  surrounding  CV2  form  a 
potent  defense.] 

6.  Do  you  wish  to  revise  the  planned  movement  for  your  own  course 
of  action  or  the  assumed  enemy  response?  If  so,  make  the 
desired  revisions. 
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DISPLAY  8:  WORSE-CASE  ENEMY  MOVEMENTS,  REVISED  AIRSTRIKE 


IWOUST^tftl_ 


DISPLAY  9:  FORCE  UNIT  LOCATIONS  FOLLOWING  WORSE-CASE  ENEMY  MOVEMENTS,  REVISED  AIRSTRIKE 


[The  commander  is  not  now  so  concerned  about  an  attack  by 
Orange  alone,  but  he  is  concerned  about  the  threat  of  Red 
attack.  He  therefore  traces  out  the  movement  anticipated  by  Red 
in  the  event  of  a Red  attack,  as  shown  in  Display  10.  Movement 
assumptions  are: 

— Red  and  Orange  launch  a coordinated  attack  against  the  task 
force 

— Red  attacks  CV1  with  its  three  northernmost  ships  (CC,  DD,  and 
DLG) 

— Red  attacks  CV2  with  its  remaining  ships  and  submarines  (2xSG, 
DU,  DDG) 

— Red  sends  its  long  range  bomber  (TU20)  located  at  Orange  Port 
towards  CV2  and  attacks  from  the  southwest 

— Orange  launches  a massive  air  attack  (6xMIC21 , IL28)  and 
engages  CV2  from  the  south  and  east 

— Orange  reinforces  ONRODA  with  aircraft  from  Pier  City  and 
Orange  Port  and  attempts  to  return  surviving  aircraft  from 
attack  on  Grey  as  in  previous  scenario 

Positions  of  opposing  forces  (accounting  for  attrition  of  Orange 
aircraft  from  attack  on  Grey) , as  indicated  by  the  simulation 
board,  are  shown  in  Display  11. 

A nominal  estimate  for  Orange  losses  and  minimum,  nominal,  and 
worse-case  Blue  losses  are  generated  using  the  outcome  estimation 
logic  of  the  simulation  board.  Estimated  losses  are: 

ESTIMATED  ENGAGEMENT  LOSSES— AIRSTRIKE 
ASSUMING  WORSE-CASE  ENEMY  RESPONSE 

Orange:  7xMIG21,  3xMIG19,  4xSU7 , 2xIL28,  lxTU16 

Blue:  Aircraft Ships 

Minimum:  0.3xF14,  3xA7,  A6  0.3xDD,  0 . lxDE 
Nominal:  0.67xF14,  3.2xA7,  I.lxDD,  0.4xDE,  0.3xCV 
1 . 2xA6 , 0.4SH 

Maximum:  3xF14,  4xA7 , 3.4xDD,  1.4xDE,  l.lxCV 

2xA6 , 2xSH 

It  is  clear  from  the  magnitude  of  estimated  losses  that  the 
task  force  is  vulnerable  to  a coordinated  Orange/Red  attack. 

Most  of  the  estimated  damage  is  due  to  the  submarine  attack  on 
CV2. 1 

7.  Bear  in  mind  that  the  results  produced  by  the  simulation  board 
are  accurate  only  to  the  extent  that  the  board  reflects  your 
current  situation.  Taking  into  account  all  aspects  of  your 
situation,  evaluate  the  alternative  assuming  the  worse  case 
enemy  response  according  to  each  of  the  outcome  measures. 
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Revised  worse-case  enemy  movements. 


A1 ternative: 


Airstrike  of  ONRODA  followed  with  Orange/ 
ONRODA  bl ockade--worse-case  enemy  response 


Outcome  Measure 

Enemy  capability  to  launch 
sorties  against  Grey  following 
Blue  action 

Risk  of  Blue/Red  War 

Own  equipment  damaged  or 
destroyed 


Evaluation 

Orange  capability  reduced 
but  they  have  enough  air 
power  to  launch  another 
attack 

Very  high 

Very  high.  CV2  almost  cer- 
tain to  be  lost,  30%  chance 
of  losing  CV1. 


D.  Critical  Uncertainty  Identification 

I.  If  a significant  difference  exists  between  the  desirability  of 
the  two  scenarios,  what  event  or  events  are  most  responsible 
for  the  difference? 


Red  attacks  task  force. 

E.  Alternative  Generation 


1.  Are  there  other  alternative  courses  of  action  that  you  wish  to 
explore?  If  so,  which? 

destroy  ONRODA  airfield  with  naval  gunfire 

amphibious  landing 

xxx  Grey/ONRODA  air  blockade 
modification  of  airstrike  alternative 

F.  Alternative  Evaluation — Blockade 

1.  Trace  out  the  planned  movement  for  Grey/ONRODA  air  blockade  on 
the  simulation  board. 


Display  12  shows  the  planned  movement  traced  out  bv  the  commander. 

Movement  assumptions  are: 

— Air  defense  unit  (lxF14)  returned  to  CV1  and  other  air  defense 
unit  (1xF14)  returned  to  CV2 

— CV1  and  support  ships  moved  east  to  a position  50-100  nmi  off 
Grey  coast.  Air  blockade  unit  (lxF14,  2xA7)  sets  up  secondary 
CA1  100-150  nmi  off  Grey  Port  to  deter  Orange  air  strikes 

— CV2  moved  southeast  to  a position  midway  between  ONRODA  and 
Industrial  City.  Air  blockade  unit  (lxF14,  2xA7)  sets  up  pri- 
mary CAP  50-100  nmi  off  ONRODA. 

— DDG  and  CG  positioned  between  ONRODA  and  Grey  coast.  DLG 
engages  Orange  MB  to  northwest  of  ONRODA.  DD  and  CLG  posi- 
tioned between  CV1  and  CV2. 
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DISPLAY  12:  PLANNED  MOVEMENT,  BLOCKADE 


2.  Trace  out  anticipated  enemy  response  movement  on  simulation 
board. 

Force  movement  anticipated  for  Red  and  Orange  as  input  by  the 
commander  is  shown  in  Display  13.  Movement  assumptions  are: 

— Neither  Red  nor  Orange  directly  attacks  the  task  force. 

— Orange  attempts  to  break  Blue  blockade  in  order  to  attack 
Grey  Port  (4xMIC21,  lxIL28) 

— Orange  surface  fleet  engages  primary  Blue  air  blockade  (2xMB, 
2xTB)  and  remainder  (2xTB,  lxMB,  2xDD)  engages  Grey  Port  sur- 
face defense  (lxDD) 

— Air  reinforcements  (2xMIG21,  2xMIG19)  are  brought  to  ONRODA 
from  Pier  City  and  Orange  Port. 

The  simulation  board  indicates  that  the  Orange  attack  on  Grey 
Port  is  only  partially  successful  at  penetrating  the  Blue  block- 
ade. With  the  help  of  their  missile  and  torpedo  boats.  Orange 
planes  successfully  pass  around  the  first  blockade;  however,  when 
the  force  ecounters  the  second  blockade,  part  of  the  attack  force 
is  detained  (2xMIG21) . The  remaining  planes  reach  Grey  Port  and 
engage  the  Grey  base  defense  (2xA4,  lxFA) . 

Outcome  simulation  logic  estimates  the  following  losses: 

ESTIMATED  ENGAGEMENT  LOSSES— 

BLOCKADE  UNDER  ANTICIPATED  ENEMY  RESPONSE 

Orange:  8xMIG21,  2xMIG19,  4xSU7,  2xIL28,  lxTU16 


Blue:  Aircraft  Ships 

Minimum : 0 0 
Nominal:  0.3xA7  0 
Maximum:  2xA7  0 


Blue  air  losses  are  expected  as  a result  of  the  confrontation  at 
the  second  blockade.  All  of  the  Orange  aircraft  attempting  the 
attack  through  the  Blue  blockade  are  destroyed  or  are  unable  to 
return  to  ONRODA. 

3.  Do  you  wish  to  revise  the  planned  movement  for  this  course 
of  action  or  the  assumed  enemy  response?  If  so,  make  the 
desired  revisions. 

No. 

4.  Bear  in  mind  that  the  results  produced  by  the  simulation 
board  are  accurate  only  to  the  extent  that  the  board  reflects 
your  current  situation.  Taking  into  account  all  aspects  of 
your  situation,  evaluate  the  alternative  according  to  each 
of  the  outcome  measures. 
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DISPLAY  13:  ANTICIPATED  ENEMY  MOVEMENTS,  BLOCKADE 


Alternative:  Grey/OkRODA  air  blockade--anticipated  enemy  response. 


Outcome  Measure 

Enemy  capability  to  launch 
sorties  against  Grey  follow- 
ing Blue  Action 

Risk  of  Blue/Red  war 

Own  equipment  damaged  or 
destroyed 


Orange  can  penetrate  blockade 
but  incurs  heavy  losses  each 
time  they  try 

Moderate 

Low 


5.  Taking  thepointof  view  of  the  enemy  commander,  trace  out 
conceivable  enemy  courses  of  action  that  make  the  alternative 
look  very  bad. 


Again,  the  main  concern  is  a Red  attack.  The  attack  pattern 

input  is  shown  in  Display  14.  Movement  assumptions  are: 

— Red  and  Orange  launch  a coordinated  attack  ag.-'nst  CV2 

— Red  surface  units  (CG,  2xDD,  2xSG,  DLG,  DDG)  and  air  unit 
(lxTU20)  surround  CV2 

— Orange  fighter  attack  (3xMIG21,  lxIL28)  engages  CVs  from  the 

southwest  while  combined  surface/air  attack  (AxMB,  2xTB,  lxMIG21) 
engages  the  Blue  destroyer  (DLG)  from  the  south. 

— As  diversionary  measures.  Orange  sends  two  torpedo  boards  (2xTB) 
to  the  northeast  side  of  ONRODA  and  two  destroyers  to  engage 
the  Grey  Port  surface  defense  (lxDD) . 

Outcomes  estimates  are  given  below.  In  this  case,  simulation 

indicates  significant  expected  losses  for  Red. 


ESTIMATED  ENGAGEMENT  LOSSES— 
BLOCKADE  UNDER  WORSE-CASE  ENEMY  RESPONSE  //I 


Orange: 

Red: 

Blue: 


4xMIG21 , 2xMIG19 , 4xSU7 , 2x1628,  lxTU16 
0.8xCG,  1.6xDD,  0.8xDDG 
Aircraft Ships 


Minimum:  0 

Nominal:  0.3xF14,  A6,  SH3 
Maximum:  lxF14,  A6,  SH 


0 

0 . 2xCV,  0 . 2xCV,  0 . 2xDE,  3xDD 
lxCV,  DE,  3xDD 


Much  higher  losses  for  Red  and  lower  losses  for  Blue  result  under 
the  blockade  alternative  because  of  the  positioning  of  the  forces. 
Under  the  blockade.  Blue  positions  its  high-power  surface  units 
(DDG,  CG,  CLG)  in  locations  where  they  can  be  used  as  defense. 
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Because  of  this.  Red  must  devote  all  of  its  forces  to  CV2,  rather 
than  directing  some  to  CV1  to  prevent  the  two  carrier  groups  from 
merging.  Because  it  is  not  under  direct  attack  and  because  of  its 
proximity  to  CV2,  CV1  is  able  to  come  to  the  assistance  of  CV2. 

The  attacking  Red  units  must  retreat. 

16.  Do  you  wish  to  revise  the  planned  movement  for  this  course  of 
action  or  the  assumed  enemy  response?  If  so,  make  the 
desired  revisions. 

No 

17.  Bear  in  mind  that  the  results  produced  by  the  simulation 
boards  are  accurate  only  to  the  extent  that  the  board  reflects 
your  current  situation.  Taking  into  account  all  aspects  of 
your  situation,  evaluate  the  alternative  according  to  each  of 
the  outcome  measures. 


Alternative:  Grey/ONRODA  air  blockade--worse-case  enemy  response. 


Outcome  Measure 

Enemy  capability  to  launch 
sorties  against  Grey  following 
Blue  action 

Risk  of  Blue/Red  war 

Own  equipment  damaged  or 
destroyed 


Evaluation 

Red  will  be  defeated.  Orange 
can  penetrate  blockade  but 
inours  heavy  losses  each  time 
they  try 

High 

Moderate 


G.  Critical  Uncertainty  Identification 

1.  If  a significant  difference  exists  between  the  desirability 
of  the  two  scenarios,  what  event  or  events  are  most  respons 
ible  for  that  difference? 

Red  attacks  task  force 

H.  Alternative  Generation 

1.  Are  there  other  alternative  courses  of  action  that  you  wish 
to  explore.  If  so,  which? 

destroy  0NR00A  airfield  with  naval  gunfire 

amphibious  landing 

Grey/ONRODA  air  blockade 

Modification  of  airstrike  alternative 
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I.  Sequence  and  Cause/Effect  Relationships 


1.  Which  of  the  following  alternatives  are  still  under  con- 
sideration for  accomplishing  the  mission? 

xxx  airstrike  of  ONRODA  followed  with  Orange/ONRODA 
blockade 

xxx  Grey/ONRODA  air  blockade 

Destroy  ONRODA  airfield  with  naval  gunfire 

__  amphibious  landing 

2.  Critical  uncertain  events  which  have  been  identified  are: 
--Red  attacks  the  task  force 

Do  you  wish  to  add  any  others?  If  so,  what? 

None 

3.  Decisions,  critical  uncertainties,  and  outcome  measures  are 
represented  below.  Draw  arrows  between  the  boxes  so  as  to 
describe  the  direction  of  influences  (i.e.,  construct  the 
influence  diagram) . 


J.  Preliminary  Decision  Tree  Generation 

[The  computer  checks  to  see  if  the  influence  diagram  is  a deci- 
sion network.  If  so,  the  decision  tree  implied  is  automatically 
constructed  (Display  15).] 
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Red  attack 


DISPLAY  15:  PRELIMINARY  DECISION  TREE  STRUCTURE 


4.  THE  EXPANSION  PHASE 


Decision  structuring  has  been  defined  as  the  process  of  identifying  and 
organizing  the  factors  of  a decision  into  a decision  model.  In  practice, 
identifying  and  characterizing  a complete  list  of  decision  factors  is  more 
difficult  and  requires  far  more  analysis  time  than  does  organizing  those 
factors  into  a decision  model. 

There  is  good  reason  for  the  emphasis  of  decision  structuring  on  the 
search  for  relevant  decision  factors.  It  is  very  important  to  account  for 
of  the  major  factors  that  affect  the  decision  in  the  decision  model. 
Sensitivity  studies  of  the  models  developed  in  decision  analysis 
applications  show  that  small  errors  in  the  subjective  inputs  or  the 
mathematical  relationships  assumed  among  variables  will  usually  not  change 
the  conclusions  of  the  analysis.  On  the  other  hand,  omitting  some  important 
variable  from  the  decision  model  can  easily  affect  the  analysis  to  the  point 
that  a suboptimal  course  of  action  would  be  indicated. 

Earlier  phases  of  the  SRI  research,  in  which  experimental  decision 
structuring  sessions  were  conducted,  pointed  out  that  much  of  the  effort 
devoted  to  structuring  is  focused  on  a search  for  and  delineation  of  two 
types  of  decision  factors:  uncertain  events  that  may  impact  the  outcomes  to 
a course  of  action  and  new  alternatives  or  modifications  to  old  alternatives 
that  aleviate  the  adverse  consequences  of  uncertain  events.  An  analysis  of 
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these  structuring  sessions  indicated  that  the  discovery  of  a new  uncertain 
event  with  potential  adverse  consequences  often  leads  spontaneously  to  the 
generation  of  a new  or  modified  alternative.  It  was  concluded,  therefore, 
that  the  single,  most  important  aid  for  decision  structuring  would  be  a 
process  for  enhancing  a decision  maker's  ability  to  identify  important 
uncertain  events  that  have  been  previously  overlooked.  The  expansion  phase 
is  a systematic  process  for  analyzing  an  existing  decision  model  to  identify 
sensitive  areas  where  the  discovery  and  addition  of  overlooked  uncertain 
events  is  most  likely  to  increase  decision  model  reliability. 

4.  1 Concept  of  Operation 

This  section  is  meant  to  provide  only  a brief  summary  and  an  intuitive 
interpretation  of  the  process  by  which  the  expansion  phase  identifies 
important  decision  model  areas  that  require  more  detail.  A complete 
description  of  the  theory  underlying  the  expansion  algorithm  is  contained  in 
a previous  report  [12]. 

Figure  13  illustrates  a highly  simplified  decision  tree  for 
representing  the  example  decision  situation  discussed  in  Section  1.3.  A TFC 
has  a choice  between  two  actions  for  neutralizing  air  power  on  ONRODA 
Island.  He  can  conduct  an  airstrike  against  the  Island  and  follow  up  with  a 
blockade  to  prevent  resupply,  or  he  can  situate  his  carriers  between  ONRODA 
and  Grey  and  set  up  a protective  blockade  to  prevent  the  enemy  from  reaching 
its  target.  The  only  uncertainty  represented  in  the  model  is  the 
possibility  of  a Red  retaliatory  attack  against  the  task  force.  Notice  from 
the  assumed  probabilities  under  the  branches  representing  possible  Red 
reactions,  that  a Red  retaliatory  attack  is  estimated  to  be  four  times  more 
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EXPANSION  AID  — CONCEPT  OF  OPERATION 
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Figure  13  A SIMPLE  DECISION  TREE 


likely  if  the  airstrike  alternative  is  chosen  than  if  the  blockade 
alternative  is  chosen. 

To  understand  the  concept  of  operation  underlying  the  expansion 
algorithm,  assume  that  the  commander,  or  someone  on  his  staff,  supplies 
relative  value  estimates  for  each  path  through  the  tree  (shown  at  the  right 
of  the  diagram)  so  that  it  may  be  solved  to  indicate  an  alternative  with 
highest  expected  value.  With  the  sample  numbers  shown,  airstrike  is 
slightly  preferable. 

The  difficulty  with  this  simple  analysis  is  in  assessing  reliable  value 
estimates.  Although  a number  of  approaches  exist  for  assessing  values,  they 
all  require  the  subject  to  consider  outcomes  at  a level  of  detail 
commensurate  with  the  structure  of  the  decision  tree.  One  such  approach 
would  be  to  ask  the  subject  to  imagine  that  the  decisions  and  events 
corresponding  to  each  path  through  the  tree  occur,  to  think  about  the 
outcomes  that  would  be  implied,  and  then  to  estimate  (usually  through 
trade-off  questions)  relative  preferences  for  those  outcomes.  The  tree  in 
Figure  13  is  so  simple  that  it  fails  to  specify  in  sufficient  detail  the 
outcomes  corresponding  to  each  path.  For  this  reason,  the  decision  maker  is 
likely  to  feel  very  uncertain  about  what  value  numbers  to  assign. 

In  principle,  the  value  numbers  that  should  be  assigned  in  the  simple 
decision  tree  of  Figure  13  could  be  obtained  by  constructing  a much  more 
detailed  decision  tree  containing  all  of  the  factors  that,  because  they  are 
omitted  from  Figure  13,  make  value  assessment  difficult.  To  obtain  the 
proper  value  number  associated,  for  example,  with  the  topmost  path  through 
the  tree  (airstrike  followed  by  Red  attack),  the  expectation  (average 
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weighted  by  probabilities)  would  be  taken  over  all  of  the  values  assigned  in 
the  more  detailed  tree  to  paths  that  begin  with  the  action  airstrike  and 


contain  the  event  Red  attack.  The  decision  maker's  uncertainty  over  what 
values  to  assign  in  the  simple  tree  can  be  interpretted  as  due  to  the  fact 
that  the  more  detailed  tree  has  not  been  constructed  and  analyzed  to  obtain 
appropriate  values  in  this  way. 

Figure  14  again  shows  the  simple  decision  tree,  this  time  with 
illustrations  of  the  probability  distributions  that  might  represent  a 
decision  maker's  subjective  uncertainty  about  what  expected  values  might  be 
obtained  if  the  detailed  decision  tree  were  constructed.  As  can  be  deduced 
from  the  widths  of  the  distributions,  the  decision  maker  feels  more 
uncertain  about  estimates  for  the  value  of  scenarios  where  a Red  attack 
occurs  than  he  does  about  estimates  for  the  value  of  scenarios  where  a Red 
attack  does  not  occur.  If  the  user  feels  very  uncertain  about  the  value  at 
the  end  of  a path,  then  adding  more  detail  to  the  path  can  be  expected  to 
resolve  considerable  uncertainty.  Conversely,  if  the  user  does  not  feel 
much  uncertainty,  then  adding  more  detail  is  not  likely  to  resolve  much 
uncertainty. 

If  a user  wanted  to  improve  the  reliability  of  a simple  decision  tree, 
it  might  at  first  seem  that  more  detail  should  be  added  to  the  paths  where 
outcome  value  uncertainty  is  the  greatest.  There  is,  however,  another 
i>ns tdcrat ion — the  amount  of  change  in  the  value  estimate  that  would  be 
pit  rod  to  switch  the  decision.  The  dashed  lines  and  arrows  in  Figure  14 
•mounts  by  which  the  estimated  value  for  each  path  would  have  to 
i - ln,>  the  other  values  constant)  to  produce  a switch  in  the 
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Figure  14  DECISION  TREE  AND  DECISION  MAKER'S  SUBJECTIVE  UNCERTAINTY  ABOUT  OUTCOME  VALUE  ESTIMATES 


alternative  with  the  highest  expected  value.  It  can  be  seen  that  resolving 
uncertainty  in  the  value  associated  with  the  path  airstrike  followed  by  no 
Red  attack  is  most  likely  to  indicate  that  the  simple  model  has  incorrectly 
identified  the  optimal  decision.  Intuitively,  then,  this  is  an  important 
scenario  to  structure  further  so  as  to  improve  the  reliability  of  the  model. 

The  expansion  phase  of  the  structuring  process  operates  according  to 
this  concept.  A computer  program  calculates  the  value  of  resolving  outcome 
uncertainty  along  each  path  through  a current  decision  tree  model.  The  path 
with  the  highest  value  of  eliminating  outcome  uncertainty  is  specified  and 
used  to  generate  questions  to  identify  important  uncertain  events  that  have 
been  omitted  from  the  tree. 

4. 2 Expansion  Phase  Functions 

Figure  15  shows  the  functions  executed  in  the  expansion  phase.  The 
figure  distinguishes  between  those  structuring  functions  that  require  human 
judgment  and,  therefore,  must  be  performed  by  the  user  (shown  in  boxes  with 
right-angled  corners)  and  those  functions  that  a computer  can  perform 
independently  (shown  in  boxes  with  rounded  corners).  As  described  earlier, 
because  outcome  calculators  and  a value  model  are  not  unique  to  a specific 
decision,  they  can  be  developed  before  the  time  they  are  needed  and  supplied 
to  the  structuring  process  as  prestructured  models.  Prestructured  models 
are  represented  in  the  flowchart  by  six-sided  boxes. 

The  process  starts  at  the  top  of  the  flowchart.  It  is  assumed  that  a 
preliminary  decision  tree  has  been  specified  and  provided  as  input.  Hie 
first  step  is  to  estimate  and  assess  ranges  of  uncertainty  for  the  outcomes 
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expected  for  each  path  through  the  preliminary  decision  tree.  Outcome 
calculators  may  be  used  to  help  generate  these  estimates,  but  it  is 
important  that  the  uncertainty  ranges  used  accurately  reflect  the 
commander's  confidence  in  the  estimates. 

Tiie  second  step  is  to  solve  the  decision  tree  and  to  compute  tiie  value 
of  resolving  residual  outcome  uncertainty  at  each  node  in  the  tree,  thereby 
indicating  areas  needing  more  detail.  Once  the  areas  for  expansion  are 
Identified,  the  third  step  is  to  generate  questions  designed  to  identify 
uncertain  events  to  be  included  in  the  model  areas  needing  expansion.  The 
method  for  generating  the  questions  is  to  compute  the  decision  sensitivity 
of  each  outcome  variable.  The  outcome  variable  to  which  tiie  decision  is 
most  sensitive  forms  the  focal  point  for  tiie  question.  For  example,  suppose 
the  most  sensitive  outcome  variable  for  the  scenario  corresponding  to  the 
path  through  the  tree  with  tiie  highest  value  of  further  modeling  was 
aircraft  losses.  Tiie  prompting  question  would  ask  the  decision  maker  to 
imagine  that  the  identified  scenario  were  to  occur,  and  then  to  think  of 
events  that  could  cause  the  number  of  aircraft  losses  to  be  significantly 
different  from  his  original  estimate.  The  exact  manner  in  which  tills 
technique  is  implemented  to  encourage  the  generation  of  overlooked  events  is 
illustrated  in  tiie  exhibit  in  Section  4.3. 

Events  that  are  proposed  for  addition  to  the  decision  tree  are  tested 
in  Step  4 to  check  whether  they  might  have  sufficient  impact  on  the  model  to 
influence  results.  Tiie  reason  for  testing  newly  identified  factors  before 
including  them  in  the  model  is  to  avoid  modeling  effort  in  those  cases  where 
tiie  effort  would  not  affect  the  recommended  decision  strategy.  Tiie  method 
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of  testing  is  to  assess  a rough  estimate  of  the  probability  that  the  newly 
identified  event  will  occur.  The  probability  that  would  be  required  in 
order  for  the  event's  inclusion  in  the  model  to  affect  the  decision  is  then 
calculated.  If  the  estimated  probability  is  greater  than  or  equal  to  the 
required  probability,  then  the  fifth  step,  expand  the  model  to  include  the 
new  event,  is  executed.  Since,  in  practice,  new  alternatives  are  sometimes 
generated  in  response  to  a newly  proposed  event,  any  modeling  effort 
conducted  in  Step  5 will  also  be  directed  toward  including  newly  identified 
alternatives  in  the  decision  tree  structure. 

The  arrow  in  Figure  15  from  Step  5 back  to  Step  1 indicates  that  the 
expansion  phase  is  an  iterative  process.  The  steps  of  the  process  are 
repeated  until  the  analysis  indicates  a value  of  additional  expansion 
sufficiently  low  enough  that  the  resource  costs  of  further  analysis  are  not 
warranted . 

A. 3 Aids  for  the  Expansion  Phase 

The  primary  aid  for  the  expansion  phase  is  an  interactive  program  for 
decision  tree  expansion  and  analysis.  This  program  may  be  used  with  or 
without  auxiliarly  aids  for  simplifying  the  generation  of  required  inputs. 
Auxiliary  aids  that  can  be  used  with  the  program  include  outcome  calculators 
and  aids  for  probability  encoding  and  value  model  development. 

The  initial  implementation  of  the  program  for  decision  tree  expansion 
and  analysis  was  undertaken  using  conventional  computer  terminals  with 
limited  graphics  capabilities  located  at  SRI  International.  The  program  was 
then  transferred  to  the  ODA  test  bed.  Recent  programming  effort  has 


concentrated  on  developing  graphics  that  make  use  of  the  advanced  display 
capabilities  at  the  test  bed,  streamlining  programing  to  improve  convenience 
and  flexibility,  and  making  minor  algorithm  modifications  to  improve 
operation.  A technical  description  of  the  program  has  been  presented  in  a 
previous  report  [12]  and  will  not  be  repeated  here.  The  following 
subs<  ctions  briefly  summarize  the  current  program  status  and  recent 
improvements . 

4.3.1  Graphics  Implementation 


Delays  in  the  installation  of  graphics  hardware  and  software  and  some 
recent  equipment  failures  at  the  ODA  test  bed  have  made  it  impossible  to 
complete  implementation  of  newly  designed  I/O  graphics  during  this  phase  of 
the  research  program-  Nevertheless,  considerable  progress  has  been  made. 

The  majority  of  the  graphics  capabilities  are  now  operational.  Decision 
trees  may  be  drawn  and  displayed  in  a natural  manner;  string  and  box  plots 
may  be  used  to  display  the  minimum,  maximum,  and  nominal  estimates  for 
uncertain  variables.  Graphics  color  is  used  to  increase  the  quantity  of 
information  that  can  be  conveniently  displayed.  Color  reproductions  of 
photograph  , howing  the  display  capabilities  of  the  program  as  currently 
available  at  the  test  bed,  were  included  in  a recent  SRI  status  report  117]. 


The  current  graphics  implementation  is  incomplete  in  two  respects. 
First,  not  all  graphics  manipulation  functions  have  been  implemented;  so, 
the  full  flexibility  of  displays  is  not  yet  available.  Second,  the  desired 
format  for  specific  displays  has  not  been  fully  implemented.  Figures  4 
through  6 in  Section  2 and  the  displays  in  the  exhibit  in  Section  4.4  show 
the  format  for  specific  displays  used  in  the  expansion  phase.  Except  for 
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the  use  of  color  In  the  actual  displays,  the  black  and  white  Illustrations 
show  the  current  graphics  design  as  it  will  appear  when  implementation  is 
complete . 

4.3.2  Improved  System  Efficiency  and  Flexibility 

A number  of  important  changes  have  been  made  to  improve  the  efficiency 
and  flexibility  of  the  expansion  program.  The  use  of  multiple  workspaces 
enables  the  parallel  processing  capability  of  the  DAISY  software  system  at 
the  test  bed  to  be  used.  Data  are  now  referenced  and  retrieved  with 
consistent  bookkeeping  subroutines.  These  two  changes  significantly 
accelerate  program  execution. 

Program  structure  has  been  redesigned  to  improve  flexibility.  The  user 
is  no  longer  forced  to  execute  the  various  steps  of  the  algorithm  in  a 
strict  sequential  fashion.  Program  modules  have  been  restructured  to  allow 
the  user  to  freely  move  from  one  operation  to  another.  This  new  design 
allows  the  user  inputs  made  in  previous  steps  to  be  more  easily  modified  and 
permits  larger  or  more  tedious  problems  to  be  analyzed  in  parts,  stored,  and 
then  later  retrieved  for  further  analysis. 

4. 3. 3 Redesign  of  Probabilistic  Processing  and  Event  Testing  Subroutines 

The  subroutines  used  for  probabilistic  processing  have  been  redesigned. 
Uncertainties  are  now  approximated  using  beta  distributions.  Beta 
distributions  are  more  general  than  normal  distributions  in  that  they  can  be 
used  to  represent  skewed  (nonsyrametr ic)  as  well  as  symmetric  distributions. 
The  capability  of  representing  distributions  that  are  skewed  is  important 
for  decision  problems  containing  uncertainties  with  expected  deviations  that 
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are  larger  on  one  side  of  the  nominal  estimate  than  the  other. 

Uncertainties  over  own  force  losses,  for  example,  may  be  skewed  to  one  side 
or  the  other  of  the  nominal  estimate,  depending  on  circumstances. 

Hie  algorithm  used  for  testing  new  factors  proposed  for  addition  to  the 
decision  tree  has  also  been  redesigned.  The  new  design  is  illustrated  in 
the  exhibit  in  Section  4.4.  It  requires  fewer  quantitative  inputs  and 
speeds  up  the  testing  function. 

4 . 4 Sample  Application  of  the  Expansion  Aid 

Exhibit  2 illustrates  the  use  of  the  automated  aid  for  the  expansion 
phase  through  the  continuation  of  the  sample  application  description  begun 
in  Exhibit  1,  Section  3.3.  The  computer  is  called  on  to  supply  considerably 
more  processing  support  in  the  expansion  phase  than  in  the  preliminary 
structuring  phase,  and  therefore  more  quantitative  inputs  are  required.  A 
major  objective  of  the  design  of  the  expansion  algorithm  has  been  to  make 
use  of  approximation  so  as  to  streamline  the  inputting  of  data  to  keep  time 
and  data  requirements  to  an  acceptable  level.  As  in  Exhibit  1,  the 
description  of  the  process  has  been  outlined  for  clarification  according  to 
the  principal  structuring  functions. 
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EXHIBIT  2:  SAMPLE  APPLICATION— EXPANSION  PHASE 


A.  Estimate  Event  Probabilities,  Expected  Outcomes,  and 
Ranges  of  Uncertainty 

If  the  user  elects  to  continue  the  structuring  process  into  the 
expansion  phase,  he  must  first  provide  the  quantitative  inputs  to  the 
preliminary  decision  tree.  This  requires  the  assessment  of 
probabilities  for  each  uncertain  event  in  the  tree  and  the  assignment  of 
values  (utilities)  for  representing  the  decision  maker's  preferences  for 
the  scenarios  represented  by  the  paths  through  the  tree.  The 
recommended  method  for  assigning  values  consists  of  three  steps:  (1) 
define  outcome  variables  for  quantifying  the  outcome  measures  developed 
in  the  preliminary  structuring  phase,  (2)  estimate  each  outcome  variable 
for  each  path  through  the  tree  (e.g.,  by  hand  or  using  prestructured 
outcome  calculators),  (3)  convert  outcome  estimates  for  each  path  to  a 
single  number  (representing  overall  preference  for  that  path)  using  a 
value  model.  Specific  aids  for  assessing  probabilities  and  developing 
values  are  not  provided  with  the  current  aid.  Several  computer  programs 
for  assisting  the  elicitation  of  probabilities  and  developing  value 
models  are  already  in  existence  [18,19]. 

1.  Probabilities  of  Critical  Uncertainties 


The  preliminary  decision  tree  is  shown  in  Display  15  of  Exhibit  1. 

In  this  example  application,  the  probability  of  a Red  attack  under 
the  airstrike  option  was  estimated  to  be  0.04.  The  probability  of  a 
Red  attack  under  the  blockade  option  was  estimated  to  be  0.01. 

2.  Outcome  Variables  and  Value  Model 

In  the  example  application,  outcome  variables  were  chosen  to 
correspond  to  the  measures  defined  in  the  preliminary  structuring 
phase.  For  own  equipment  damaged,  outcome  variables  were  chosen  to 
represent  the  fraction  of  damage  to  critical  aircraft  and  ships. 

The  outcome  variable  for  enemy  sortie  capability  was  chosen  to  be 
the  number  of  potential  Orange  sorties  that  could  be  launched  from 
ONRODA  in  a three-month  period  following  the  Blue  action.  The 
outcome  variable  for  the  risk  of  Blue/Red  war  was  chosen  to  be  the 
commander's  subjectivi  probability  that  Red  would  directly  attack 
Blue  nation. 
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A linear,  additive  value  model  was  used  to  indicate  the  relative 
importance  of  outcome  variables: 

V = - x - 7y  - 0. 003z  - 91 2w 

where 


x = 6[F14s  lost]  + 3 [A 6s  lost]  + [A 7s  lost] 

represents  aircraft  losses  in  A7  aircraft  equivalents,  and 

y = 2.5[CG  cap.  lost]  + 25[CV  cap.  lost] 

gives  ship  losses  in  DLG  ship  equivalents.  The  number  of  potential 
Orange  sorties  from  ONRODA  during  the  three-month  period  following 
the  Blue  action  is  z,  and  w is  the  probability  that  Blue-Red  war 
results . 

3.  Outcome  Estimates 

Display  16  illustrates  the  use  of  the  computer  aid  for  eliciting 
outcome  estimates.  The  computer  asks  the  user  to  enter  best,  low, 
and  high  estimates  for  the  expected  value  of  each  outcome  variable. 
Estimates  are  requested  for  each  path  through  the  tree.  The  path 
addressed  in  the  display  is  airstrike  followed  by  no  Red  attack. 

User  inputs  are  underlined. 

B-  Identification  of  Model  Areas  for  Expansion 

As  the  outcome  estimates  are  input,  the  computer  fits  a probability 
distribution  (based  on  beta  functions)  such  that  the  best  estimate  is 
the  median,  and  the  low  and  high  estimates  are  the  10%  and  90%  points  of 
the  distribution.  The  program  then  calculates  a distribution  on  overall 
outcome  value  for  each  path  through  the  tree,  solves  the  tree,  and  then 
calculates  the  value  of  resolving  the  uncertainty  in  outcome  value  for 
each  path  through  the  tree. 

Display  17  illustrates  the  manner  in  which  the  computer  summarizes 
results.  The  display  shows  the  decision  tree  and  the  decision  strategy 
with  the  highest  expected  value.  Although  not  visible  in  the  black  and 
white  reproduction,  the  computer  highlightes  the  path  with  the  highest 
value  of  further  modeling  by  coloring  it  red.  In  this  case,  the  path 
with  the  highest  value  of  modeling  is  airstrike  followed  by  no  Red 
attack. 
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OUTCOME  ESTIMATION 


ENTER  BEST  (50\).  LOW  (10'.),  AND  HIGH  (90‘.)  ESTIMATES  FOR  THE 
EXPECTED  VALUE  OF  EACH  OUTCOME  VARIABLE  BELOW.  ASSUME 
THAT  THE  ALTERNATIVE  CHOSEN  IS  "AIR  STRIKE"  AND  THAT  THE 
EVENT  "NO  RED  ATT"  OCCURS. 


AIR  LOSS 


F14  BEST  ESTIMATE:  4 LOW  ESTIMATE:  ]_  HIGH  ESTIMATE  9 

A6  BEST  ESTIMATE  1 LOW  ESTIMATE  0 HIGH  ESTIMATE  3 

A7  BEST  ESTIMATE.  I LOW  ESTIMATE:  0 HIGH  ESTIMATE  6 


Display  16  GRAPHICS  DISPLAY  USED  FOR  OUTCOME  ESTIMATION 


l 
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DECISION  TREE 


r-RED  ATT  ( 04)  - EV=-193  VOM=  2 


-AIR  STRIKE  ♦ EV=  -47  V0M=22 


-NO  RED  ATT ( .96)  EV=  -41  V0M=22 


EV=  -47  VOM=22 


r— RED  ATT  (.01) — EV=-144  VOM=  0 


-BLOCKADE  *EV=  -53  VOM-12 


-NO  RED  ATTf.99)  EV=  -52  V0M=12 


C.  Generate  Questions  for  Identifying  Important  Events 


To  assist  in  identifying  events  to  add  to  the  decision  tree,  the 
computer  conducts  a sensitivity  analysis  to  identify  which  outcome 
variable — aircraft  lost,  ships  lost.  Orange  sorties,  or  risk  of 
Blue/Red  war — is  most  responsible  for  the  importance  of  uncertainty 
along  the  path  with  the  highest  value  of  modeling.  In  this  case 
aircraft  lost  was  computed  to  be  the  most  sensitive  outcome  variable. 

Since  the  path  through  the  tree  with  the  highest  value  of  modeling  is 
airstrike  followed  by  no  Red  attack,  and  aircraft  lost  is  the  most 
sensitive  outcome  variable,  the  computer  generates  the  following 
question  designed  to  help  identify  missing  events: 

Suppose  .you  chose  the  alternative  "air  strike"  and 
the  event  "no  Red  attack"  occurs.  Is  there  any 
event  not  yet  included  in  the  model  that  could 
cause  "air  loss"  to  increase? 


In  this  application  the  commander  was  able  to  suggest  an  event  that 
could  lead  to  significantly  larger  aircraft  losses.  The  event  was  that 
Red  may  have  installed  sophisticated  anti-aircraft  surface-to-air  missle 
(SAM)  batteries  on  the  Island.  If  this  were  the  case,  the  commander 
reasons  that  he  would  be  likely  to  lose  more  planes  than  originally 
estimated.  He  therefore  responds  that  there  is  such  an  event.  The 
computer  then  requests  a description  and  a short  name  for  the  newly 
identified  event.  Display  18  illustrates  the  appearance  of  the  computer 
display  screen  during  this  structuring  function. 

If  the  user  had  been  unable  to  identify  an  event  that  could  cause 
aircraft  losses  to  be  significantly  higher,  the  computer  would  have 
composed  a similar  question  based  on  the  second  most  sensitive  outcome 
variable.  Additional  questions  are  composed  in  this  way  until  a new 
event  is  successfully  identified. 


D.  Event  Testing 


Events  proposed  for  addition  to  the  decision  tree  are  tested  to 
determine  if  adding  the  event  to  the  tree  might  affect  the  decision. 
Displays  19  through  21  illustrate  the  testing  function.  The  first 
question  asks  whether  the  decision  maker  would  alter  the  decision  if  he 
were  certain  that  the  event  would  occur.  If  he  would  not,  there  is  no 
point  to  adding  the  event  to  the  model.  Because  the  user  answers  "yes," 
however,  a more  sophisticated  test  is  conducted. 

The  second  test  is  to  see  if  the  event  is  sufficiently  likely  and 
important  to  affect  the  decision.  The  test  consists  of  estimating  the 
outcome  that  would  result  if  the  event  were  to  occur,  and  then  computing 
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DECISION  FACTOR  IDENTIFICATION 


SUPPOSE  YOU  CHOSE  THE  ALTERNATIVE  "AIR  STRIKE"  AND  THE  EVENT 
"NO  RED  ATT"  OCCURS  IS  THERE  ANY  EVENT  NOT  YET  INCLUDED  IN 
THE  MODEL  THAT  COULD  CAUSE.  . . 

. . "AIR  LOSS”  TO  INCREASE? 

(A  VALUE  OF  58  WOULD  BE  SUFFICIENT  TO  PRODUCE  A DECISION 
SWITCH) 

YES 

WHAT  DO  YOU  WISH  TO  CALL  THIS  EVENT? 


Display  18  DISPLAY  FOR  FACTOR  IDENTIFICATION 


N 
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DECISION  FACTOR  TESTING 


ROUGHLY.  WHAT  IS  THE  PROBABILITY  OF  THE  EVENT  "SAM  SITES"? 

OJ 

SUPPOSE  YOU  CHOSE  THE  ALTERNATIVE  "AIR  STRIKE".  IF  YOU  KNEW 
FOR  SURE  THE  EVENT  "SAM  SITES"  WOULD  OCCUR.  WOULD  YOU 
CHANGE  YOUR  BEST  ESTIMATE  FOR  THE  EXPECTED  VALUE  OF  THE 
OUTCOME  "AIR  LOSS"’ 

YES 

IF  YOU  KNEW  THE  EVENT  "SAM  SITES"  WOULD  OCCUR.  WOULD 
YOUR  REVISED  ESTIMATE  FOR  THE  EXPECTED  VALUE  OF  THE  OUTCOME 
“FI 4"  BE  MUCH  LOWER  (0.0  - 0.8),  LOWER  (0.9  - 2.3).  ABOUT  THE  SAME 
(2.4  - 6.1),  HIGHER  (6.2  - 10.2),  MUCH  HIGHER  (10.3  - 48.0)  ? 

HIGHER 


Display  20  TESTING  DISPLAY-2 


91 


DECISION  FACTOR  TESTING 


SUPPOSE  YOU  CHOSE  THE  ALTERNATIVE  "BLOCKADE"  IF  YOU  KNEW 
FOR  SURE  THAT  THE  EVENT  "SAM  SITES"  WOULD  OCCUR,  WOULD  YOU 
CHANGE  YOUR  BEST  ESTIMATE  FOR  THE  EXPECTED  VALUE  OF  THE 
OUTCOME? 


Display  21  TESTING  DISPLAY- 3 


, i 
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the  probability  the  event  would  have  to  have  for  the  commander  to  wish 
to  switch  decisions.  If,  when  the  worst  possible  outcomes  are  assumed, 
the  required  probability  of  the  event  necessary  to  cause  the  decision  to 
switch  is  less  than  the  estimated  probability  of  the  event,  then  that 
event  should  be  added  to  the  decision  model.  As  the  displays 
illustrate,  testing  the  proposed  event  in  this  application  verified  that 
it  should  be  added  to  the  decision  model. 


E.  Expanding  the  Decision  Tree 


As  in  the  preliminary  structuring  phase,  influence  diagrams  are  used  to 
simplify  adding  events  to  the  decision  tree.  The  computer  draws  the 
existing  influence  diagram  and  the  newly  proposed  event  and  asks  the 
user  to  draw  the  arrows  required  to  add  the  event  to  the  diagram.  This 
is  illustrated  in  Display  22.  The  user  reasons  that  his  decision  to 
launch  an  airstrike  or  a blockade  does  not  affect  the  probability  of  SAM 
sites,  and  whether  or  not  SAM  sites  exist  does  not  affect  the 
probability  of  a Red  attack.  Therefore,  the  correct  influence  diagram 
contains  no  additional  arrows. 

When  the  user  signals  the  program  that  he  has  completed  modifications  to 
the  influence  diagram,  the  computer  checks  to  see  that  the  diagram 
corresponds  to  a realizable  decision  tree.* 


F.  Input  Estimation  and  Evaluation  of  the  Expanded  Tree 


One  iteration  of  the  expansion  cycle  is  now  complete.  To  determine  the 
solution  to  the  expanded  model  and  the  value  of  further  modeling,  the 
user  must  supply  the  necessary  additional  probability  and  outcome 
estimates.  Display  22  shows  computer  graphics  for  eliciting  the 
additional  outcome  estimates. 

Notice  that  it  is  not  necessary  to  reassess  outcomes  for  every  path 
through  the  expanded  tree.  The  only  new  estimates  required  are  for  those 
scenarios  that  involve  an  airstrike  with  SAM  sites  assumed.  In  this 
application,  the  user  stated  that  he  did  not  believe  the  event  SAM  sites 
would  slgnficantly  affect  his  estimate  of  ship  losses,  potential  Orange 
sorties,  or  the  risk  of  Blue/Red  war.  Therefore,  revised  estimates  were 
supplied  only  for  the  outcome  variable  aircraft  losses. 

Display  23  shows  the  expanded  decision  tree.  The  preferred  alternative 
now  appears  to  be  blockade.  The  path  through  the  tree  with  the  highest 


*An  error  in  diagram  construction  or  a poorly  formulated  model  (for  example, 
an  influence  diagram  with  a loop)  must  be  reformulated  to  be  represented  as  a 
decision  tree.  The  program  checks  the  diagram  and  points  out  to  the  user  any 
reasons  that  may  prevent  the  diagram  from  being  converted  to  a decision  tree. 
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^ DECISION  FACTOR  MODELING 

ADD  THE  EVENT  "SAM  SITES"  TO  THE  INFLUENCE  DIAGRAM 


r 


value  of  further  modeling  is  now  blockade  followed  by  no  Red  attack, 
additional  structuring  activity  is  to  be  undertaken,  this  is  the 
scenario  that  should  be  expanded.  The  value  of  further  modeling, 
however,  has  been  reduced  to  about  one-half  of  the  previous  value. 
Depending  on  the  time  and  perceived  importance  of  this  decision,  the 
USer  may  choose  to  continue  or  terminate  structuring  activity  at  this 

po int  ■ 


If 


5.  CONCLUSIONS  AND  RECOMMENDATIONS 


The  principal  recommendation  from  this  phase  of  the  research  is  that 
research  should  be  continued  and  directed  toward  first  completing  and  then 
formally  testing  the  decision  structuring  aid. 

The  recommendation  for  continuing  development  is  based  both  on 
characteristics  of  the  structuring  process  that  suggest  ire  potential  value 
for  aiding  TFC  decision  making  and  on  the  results  of  preliminary 
applications.  Important  characteristics  include: 

• Flexibility  — The  aid  may  be  applied  to  virtually  any  unstructured 
decision  situation. 

• Efficiency  — The  aid  provides  tests  that  reduce  modeling  effort  by 
enabling  factors  that  do  not  affect  the  decision  to  be  identified 
before  they  are  formally  modeled. 

• Realism  — Restrictive  assumptions,  such  as  assuming  independence  of 
uncertainties  or  a normal  shape  for  probability  distributions,  are 
not  required. 

• Compatibility  — The  aid  may  be  used  with  prestructured  models  for 
estimating  decision  outcomes  to  produce  an  integrated  decision  aid. 
The  integrated  aid  speeds  up  the  structuring  of  a decision  model  and 


shows  how  outcome  calculators  may  be  employed  to  identify  an  optimal 
course  of  action. 

The  limited  experimental  applications  of  the  structuring  process 
conducted  so  far  have  used  both  subjects  who  are  familiar  with  and  who  are 
unfamiliar  with  decision  analysis.  Surprisingly,  the  decision  tree  models 
developed  during  these  applications  were  simpler  than  models  normally 
developed  through  the  standard  decision  analysis  methods.  Nevertheless,  in 
all  cases  the  models  developed  and  their  solutions  were  readily  accepted  as 
adequate  by  the  subjects.  The  time  required  was  much  less  than  that  using 
traditional  structuring  approaches. 

5.  1 Principles  for  Testing 

The  detailed  design  for  a testing  program  for  evaluating  the  SRI 
decision  structuring  aid  should  be  developed  following  completion  of 
computer  implementation.  Based  on  the  observed  characteristics  of  the 
process  and  the  results  of  preliminary  applications,  however,  several 
principles  for  testing  can  now  be  identified. 

Most  important  for  the  testing  program  will  be  specification  of  a 
criterion  for  evaluating  the  extent  to  which  the  aid  improves  decision 
making.  An  obvious  approach  is  to  design  standard  decision  problems  to  be 
used  for  testing.  Solutions  to  these  standard  problems  might  be  based  on 
the  combined  judgment  of  a panel  of  experienced  naval  commanders. 

Experiments  could  then  measure  the  extent  to  which  subjects  using  the  aid 
come  closer  to  the  prescribed  solution  than  do  subjects  not  using  the  aid. 
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There  are  a number  of  problems  with  such  an  approach.  First,  the  aid 
has  been  designed,  by  definition,  for  application  to  one-of-a-kind, 
unfamiliar  decisions  that  have  not  been  previously  structured.  Application 
to  standardized  decision  problems  may  result  in  testing  the  aid  on  a class 
of  problem  for  which  It  Is  not  Intended  to  be  used. 

An  even  more  fundamental  problem  Is  the  definition  of  a good  decision. 
According  to  the  discipline  of  decision  analysis,  a good  decision  is  one 
that  Is  consistent  with  the  decision  maker's  information  and  preferences. 

The  structuring  process  is  designed  to  help  a user  identify  a decision 
strategy  that  Is  consistent  with  his  own  subjective  preferences  and 
Information.  Thus,  successful  application  of  the  aid  can  result  in  totally 
different  decision  strategies  for  Individuals  who  have  different  information 
or  preferences.  Comparing  the  solution  indicated  by  the  structuring  aid 
with  the  "right"  solution  generated  by  a panel  of  experts  does  not, 
therefore,  directly  measure  the  aid's  ability  to  improve  a decision  maker's 
performance.  The  user  could  conceivably  believe  that  the  strategy  derived 
through  the  use  of  the  aid  was  better,  given  his  state  of  information  and 
preferences,  than  that  produced  by  the  panel  of  experts.  We  believe  that  a 
formal  decision  analysis  resulting  in  a quantitative  model  of  the  type 
produced  by  the  structuring  process  is  the  most  effective  way  to  generate  a 
consistent  decision  strategy.  Other  methods,  such  as  the  use  of  a panel  of 
experts,  result  in  well-known  group  decision-making  biases. 

Once  an  adequate  experimental  design  is  selected,  testing  should 
attempt  to  answer  several  specific  questions.  Most  Importantly,  if  the  aid 


falls  to  perform  well,  experimental  design  should  Indicate  If  the  reason  is 


due  to  the  user's  Inability  to  provide  appropriate  input  information,  if  the 
difficulty  is  a more  general  problem  with  the  decision-analysis  modeling 
approach,  if  it  is  a problem  related  to  the  specific  algorithm  utilized  in 
the  SRI  structuring  process,  or  if  it  is  a prob’  ira  with  the  computer  aid 
design. 

A variable  that  may  affect  the  value  of  the  aid  to  a user  is  the  user's 
proficiency  in  decision  analysis  techniques.  Conseqeuntly , testing  should 
attempt  to  identify  whether  the  effectiveness  of  the  aid  depends  strongly  or. 
the  user's  familiarity  with  decision  analysis.  Because  the  structuring 
process  design  lends  itself  to  a very  wide  class  of  decision  problems 
testing  should  also  attempt  to  determine  whether  specific  kinds  of  decisions 
are  more  effectively  aided  than  others.  If  this  is  the  case,  then  the 
characteristics  that  tend  to  make  the  aid  very  effective  or  ineffective 
should  be  identified. 

Ease  of  use  of  the  aid  will  be  strongly  influenced  by  input/output 
design,  especially  the  use  of  graphics.  Therefore,  testing  should  attempt 
to  identify  what  changes  in  computer  aid  design  are  needed  to  simplify  its 
use.  Finally  whatever  criterion  is  adopted  for  measuring  the  ability  of 
the  aid  to  improve  decision  making  some  attempt  should  be  made  to  assess 
the  extent  to  which  the  users  feel  that  the  aid  is  helping  them. 

Specifically  does  the  aid  help  users  to  identify  issues,  does  it  help  them 
to  evaluate  options,  and.  finally,  do  the  users  feel  as  though  the  aid  helps 
them  to  formulate  better  decision  strategies? 
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